Using Amazon Bedrock for AI Orchestration

Generative AI has been advancing at a rapid pace over the past year, and even in just the last few months, we’ve seen some remarkable breakthroughs. While the direct use of LLM models can still be tremendously effective, especially when combined with fine-tuning and Retrieval-Augmented Generation (RAG), the use of agents represents a seismic change. When using agents, orchestration is a requirement. This blog explains the importance of AI orchestration and how to use it with Amazon Bedrock.  

The Evolution of GenAI Systems

The evolution of GenAI systems typically follows this progression:

1. Using models alone
2. Enhancing models with external data via Retrieval-Augmented Generation (RAG)
3. Adding agency through agents

Agents are components that models can call to request additional information or perform actions. The capabilities of an agent are defined by the instructions given to it when it is created. For example, a simple agent instruction might be: “Search for a patient’s ID using their full name.”

If a large language model (LLM) with access to this agent is asked to perform a task that requires an ID lookup, it can delegate that part of the task to the agent. 

The Challenge of Overloaded Agents

When agents were first being developed, a common pattern was to give the agent multiple optional instructions, allowing it to perform a variety of tasks. However, it was soon noticed that this led to confusion by the LLM if the distinct instructions were not clearly separated.

In this example, we show an agent with overlapping instructions:

Agent Instructions:

1. Answer all customer questions about product features.
2. Provide troubleshooting steps for any technical issues customers describe.
3. Recommend new products to customers based on their previous purchases.
4. If a customer asks about a product feature that isn’t available, suggest a workaround or a similar product.

Resulting Agent Confusion:

If a customer asks about a feature that isn’t available, the agent must decide whether to:

• Answer the question (Instruction 1)
• Provide troubleshooting (Instruction 2)
• Recommend a new product (Instruction 3)
• Suggest a workaround, potentially all at once (Instruction 4)

In some cases, these instructions can conflict—for example, recommending a new product (Instruction 3) vs. suggesting a workaround (Instruction 4) might lead to contradictory actions.

Moving Toward Simpler Agents and AI Orchestration

To avoid this confusion, we replace one complex agent with multiple simpler agents. However, this now means that we must determine how the agents can work together. For agents, this is referred to as AI orchestration. It also means that we must have a deeper understanding of the system's underlying business logic. In other words, you can’t automate a system you don’t understand (more on that later).

Types of AI Orchestration / Workflows

The type of AI orchestration required depends heavily on the workflow it is supporting. Let’s examine two examples that require quite different types of orchestration. The first example has a highly structured orchestration because the underlying business process is also highly structured. The second example is non-deterministic because, again, the underlying business process is non-deterministic. When we say 'non-deterministic,' we mean that the conversation can go in multiple directions based on the input.

Deterministic Orchestration Example: Banking KYC Process

Workflow:

A "Know Your Customer" (KYC) process for onboarding new banking clients.

Steps:

1. Collect customer data (name, ID, address).
2. Verify ID against government databases.
3. Check for sanctions list matches.
4. Assign risk levels based on findings.
5. Initiate ongoing transaction monitoring.

Why Deterministic Orchestration Works:

• Strict regulatory requirements demand consistency.
• Predefined steps ensure compliance and auditability.
• Specialized agents handle discrete tasks (e.g., verification, risk assessment) without overlap.
• Centralized control prevents deviations from protocols.

Compare this structured workflow with a non-deterministic one.

Non-Deterministic Orchestration Example: Dynamic Customer Service Resolution

Workflow:

Resolving a complex customer complaint (e.g., a technical issue with a smart home device).

Steps:

1. AI analyzes the sentiment and urgency of the complaint.
2. Routes the issue to troubleshooting agents or escalates to human staff.
3. Iteratively diagnoses the problem using device logs, user history, and real-time API calls.
4. Proposes solutions (e.g., rebooting, firmware updates, replacement).
5. Adapts recommendations based on user feedback and success rates.

Why Non-Deterministic Orchestration Works:

• Unpredictable variables (e.g., unique device errors, user skill levels).
• At any point, the customer may become frustrated and request escalation to a human representative.
• Requires real-time data analysis and flexible decision-making.
• AI agents self-improve by learning from past interactions.

If you’ve ever worked in a call center, you know from experience how every call can be different and every customer interaction can be unpredictable.

AI Orchestration with Amazon Bedrock

Amazon Bedrock’s approach to AI agent orchestration is a significant step forward in making complex, multi-agent AI systems accessible and manageable, but it comes with both strengths and trade-offs that practitioners should carefully consider. Amazon Bedrock introduces a supervisor-based architecture in which a central “supervisor agent” coordinates a set of specialized agents, each handling distinct subtasks within a larger workflow. This design streamlines the orchestration of multi-step, domain-specific processes such as financial due diligence or customer service automation by automatically decomposing tasks, routing them to the right agents, and consolidating outputs.

However, while Amazon Bedrock reduces much of the technical overhead traditionally required for agent orchestration, such as manual coordination logic and state management, it also introduces a layer of abstraction that may limit granular control for advanced users. For example, the default ReAct orchestration method provides transparency and step-by-step reasoning but can become a bottleneck in scenarios demanding high throughput or parallelism. AWS addresses this with a custom orchestrator option, allowing users to implement bespoke orchestration strategies via AWS Lambda, thereby regaining fine-grained control over agent workflows, real-time adjustments, and state transitions. Yet, this flexibility comes at the cost of increased complexity and responsibility for the developer; bespoke strategies are almost always more complex than standard ones.

It’s important to note that Amazon Bedrock’s multi-agent pipelines are primarily designed for workflows that follow a clear, linear path. If your use case involves frequent back-and-forth or unpredictable interactions between agents, Amazon Bedrock may not be the ideal fit on its own. In such situations, combining Amazon Bedrock with open-source orchestration tools like LangGraph or CrewAI can provide the flexibility and collaborative reasoning you need, as they enable more customization. However, this comes at the cost of increased integration work and operational overhead. It is worth noting that new orchestration tools and frameworks are being introduced every month (see the new Strands Agent SDK, for example).

Orchestration Via Amazon Bedrock Flows

Amazon Bedrock offers several distinct approaches to creating orchestrations, enabling flexible ways to design and manage complex agentic workflows. One of these approaches is using Flows, which are created via the Amazon Bedrock Flow Editor, an interface that has a similar feel to the Step Function Editor. You drag and drop one or more items from the Nodes list and then draw connections between the nodes. The following flow is from an AWS video tutorial: