Getting Started with Agentic AI on AWS

This blog explores the various components and architectures that have emerged in this rapidly evolving domain, including Retrieval-Augmented Generation (RAG), knowledge bases, and tools. It delves into the intricacies of agent systems, discussing both single-agent and multi-agent approaches, and examines the delicate balance between comprehensive information gathering and token efficiency. The paper also covers key features such as prompt routing, caching, and guardrails, providing insights into their implementation and benefits. By analyzing these developments, we aim to provide a comprehensive overview of the current state of LLM interactions and their potential future directions.

The various options are discussed in turn and then code samples for each example are presented in the Appendix.

How has agentic infrastructure evolved?

The evolution of Large Language Model (LLM) interactions has seen rapid advancements in recent years. Initially, developers made direct calls to LLMs, providing prompts and receiving generated responses. This approach, while powerful, was limited by the model's training data cutoff. The introduction of Retrieval-Augmented Generation (RAG) marked a significant leap, allowing LLMs to access external, up-to-date information to enhance their responses. As RAG gained traction, the process was further streamlined with the development of managed knowledge bases and tools, automating the retrieval and integration of relevant information. The latest advancements have introduced sophisticated capabilities like Multi-Agent Orchestration, where multiple specialized AI agents collaborate to tackle complex tasks, and InlineAgents, which offer dynamic, runtime configuration of AI assistants. These developments have dramatically expanded the scope and flexibility of LLM applications, enabling more intelligent, context-aware, and adaptable AI systems.

As with most designs/architectures in our field, agents are balancing two competing pressures. Pressure one is the desire to have the most complete information so as to be able to generate the most accurate and helpful results. Pressure two is the desire to avoid token explosion (and its attendant higher cost and higher chance of hallucination) by providing too much information to the model. The evolution of agent and multi-agent approaches are attempts to balance these design pressures while simultaneously avoiding excess complexity in the actual software.

Depending on the business problem to solve, the Agent system might be a single agent or be multi-agent (sometimes abbreviated MA). In Multi-Agent Systems a group of agents, each specialized in a specific task, work together to complete a full workflow.

The latest advancements have introduced sophisticated capabilities like Multi-Agent Orchestration, where multiple specialized AI agents collaborate to tackle complex tasks, and InlineAgents, which offer dynamic, runtime configuration of AI assistants. These developments have dramatically expanded the scope and flexibility of LLM applications, enabling more intelligent, context-aware, and adaptable AI systems.

Multi-Agent Orchestration, for example, can have multiple levels: an Orchestrator Agent selecting from multiple Supervisor Agents, each of which is selecting one or more work Agents, each of which might use one or more knowledge bases and/or tools. This level of complexity can lead to difficulty with observability and evaluation.

What is an AI agent?

An AI agent is essentially just a technology system that can decide, act, and learn without constant human interaction - i.e. it is semi or fully autonomous. The system is composed of both ML/AI models and traditional software components. Typically, AI agents are used to complete specific tasks or workflows and/or do analysis and drive decisions that achieve business goals. AI Agents are typically programmed around a specific objective or set of objectives.

Most Caylent customers (although not all) will likely build agents leveraging Amazon Bedrock Agents. An Amazon Bedrock Agent consists of several key components that work together to enable complex, multi-step task execution and intelligent interactions. The main components of an agent are:

Foundation Model (FM)

The agent uses a selected foundation model as its core reasoning engine. This FM is responsible for:

• Interpreting user requests
• Breaking down tasks into logical steps
• Generating responses and follow-up actions

The FM is the component that most people associate with any GenAI system, even though it is actually only one component of such a system.

Prompts

Developers provide instructions that define the agent's purpose and guide its behavior. These instructions act as a prompt to the FM, describing what the agent is designed to do and how it should interact. These instructions can be either a system prompt or a user prompt. System prompts define the general behavior of the system, and are often lengthy and highly detailed, containing comprehensive instructions on how the AI should process information and generate responses. They may include rules for citation, formatting, and even personality traits. System prompts are often not visible to the end user but can be modified when creating a client agent.

User prompts are typically shorter and more straightforward, ranging from simple questions to more complex requests for analysis or content creation. They can vary in complexity but are generally more focused on a specific task or topic.

Action Groups / Tools

An Action Group is a component that defines specific tasks an agent can perform to assist users. It serves as a bridge between the agent's natural language understanding capabilities and the execution of concrete actions or API calls. Action Groups are composed of a set of actions, typically defined using an OpenAPI schema, and an action executor, usually implemented as an AWS Lambda function.

See Action Groups / Tools for more details

Knowledge Bases

Knowledge bases provide additional context and information to supplement the agent's responses. They allow the agent to:

• Access and query relevant data sources
• Perform Retrieval-Augmented Generation (RAG) to enhance accuracy
• Augment responses with domain-specific knowledge

See Knowledge Base for more details

Memory

Agents have both short-term and long-term memory capabilities:

• Short-term memory retains detailed information relevant to the current conversation
• Long-term memory stores important facts and summaries from previous interactions

Prompt Templates

Customizable prompt templates allow developers to fine-tune the agent's behavior at different stages of its operation, including:

• Pre-processing
• Orchestration
• Knowledge base response generation
• Post-processing

By combining these components, Amazon Bedrock Agents can orchestrate complex workflows, interact with enterprise systems, and provide intelligent, context-aware responses to user queries.

Advanced applications for agents

Knowledge Base

An Amazon Bedrock Knowledge Base is a fully managed capability that enables the implementation of Retrieval-Augmented Generation (RAG) workflows for generative AI applications. It serves as a crucial component in enhancing the responses of foundation models (FMs) by providing contextual information from an organization's private data sources. Knowledge Bases are essentially vector databases created from source documents, allowing specialized company-specific data to be made available to the Large Language Model (LLM).

To create a Knowledge Base in Amazon Bedrock, users first need to prepare their data source, which can be unstructured (such as documents in an S3 bucket) or structured (like databases in Amazon Redshift or AWS Glue Data Catalog). When setting up the Knowledge Base, users select an embedding model to convert their data into vector embeddings and choose a vector store to index these embeddings. Amazon Bedrock can automatically create and manage a vector store in Amazon OpenSearch Serverless, simplifying the setup process. The following databases may serve as the vector store for a knowledge base: Amazon OpenSearch Serverless, Amazon Aurora PostgreSQL, MongoDB Atlas, Pinecone and Redis Enterprise Cloud.

Once created, a Knowledge Base can be utilized through various operations provided by Amazon Bedrock. The Retrieve operation allows users to query the Knowledge Base and retrieve relevant information, while the RetrieveAndGenerate operation goes a step further by using the retrieved data to generate appropriate responses. For structured data sources, Amazon Bedrock Knowledge Bases can convert natural language queries into SQL queries, enabling seamless integration with existing data warehouses. This capability extends the reach of generative AI applications to include critical enterprise data without the need for extensive data migration or preprocessing.

The use of Knowledge Bases in Amazon Bedrock offers several advantages for building generative AI applications. It provides a fully managed RAG solution, automating the end-to-end workflow from data ingestion to retrieval and prompt augmentation. This approach not only improves the accuracy and relevance of AI-generated responses but also enhances transparency through source attribution, helping to minimize hallucinations. Furthermore, Knowledge Bases support multimodal data processing, allowing applications to analyze and leverage insights from both textual and visual data, thereby expanding the scope and capabilities of AI-powered solutions. 

See Knowledge Base Definition for a sample of creating a Knowledge Base.

Action Groups / Tools

Tools in the context of AI and AI-agentic systems are software that allow the AI system to perform certain defined, deterministic tasks such as interacting with external systems or running scripts. They serve as extensions to the model's capabilities, allowing it to perform tasks such as querying databases, making API calls, or accessing real-time information. Developers provide JSON schemas describing each tool's functionality and input requirements. A tool is basically the OpenAPI definition of a function call.

When creating an Action Group, developers specify the parameters that the agent needs to elicit from users to carry out actions. For example, an Action Group for a hotel booking system might include functions like "CreateBooking," "GetBooking," and "CancelBooking," each with its own set of required parameters such as check-in date, number of nights, or booking reference. The Bedrock agent uses this configuration to determine what information it needs to gather from the user through conversation. Once the necessary details are collected, the agent invokes the associated Lambda function, which contains the business logic to execute the action, such as interacting with backend systems or external APIs. This modular approach allows for flexible and extensible agent capabilities, enabling developers to create sophisticated AI assistants that can perform a wide range of tasks based on natural language inputs. 

See Action Group Definition for an example of defining an Action Group.