Build agentic retrieval on a Managed Amazon Bedrock Knowledge Base with CDK

This post builds enterprise agentic retrieval on a Managed Amazon Bedrock Knowledge Base with the AWS Cloud Development Kit (AWS CDK). To see why that’s worth building, it helps to start with where ordinary Retrieval Augmented Generation (RAG) breaks down. Most RAG systems start the same way: they embed the question, fetch the nearest chunks from one index, hand them to a model, and return the answer. One retrieval, one generation. For a large class of questions that’s the right design, and there’s no reason to build anything more complicated. It fails in a recognizable way. Ask a question whose answer spans two corpora, and a single retrieval returns half of what the model needs, which is worse than returning nothing, because the answer comes back fluent and incomplete. Ask a question where the right source depends on the question , and a single index has no mechanism to choose. Ask a question that needs a second look at the corpus after seeing the first result, and there is nowhere for that second look to happen. Enterprise agentic retrieval answers all three by putting a reasoning agent in the loop. The agent decides whether to retrieve, which knowledge base to consult, and whether to go back for more before it composes an answer. This is the direction Amazon Bedrock Knowledge Bases have evolved. With the launch of the Managed Knowledge Base, retrieval is no longer a single embed-and-fetch call. It has become an agentic capability: multi-turn planning, tool-driven retrieval, and grounded, cited answers. The AgenticRetrieveStream API offers this as a first-class service feature rather than something you assemble yourself. That works, and it also introduces a genuinely harder operational problem. A single-shot pipeline is a function you can reason about by reading it. An agent that reasons and retrieves in a loop takes a different path on every question. By default you can see neither the path it took nor whether the answer it produced was any good. Observability stops being a nice-to-have and becomes the only way to know the system works. In August 2024, Build an end-to-end RAG solution using Knowledge Bases for Amazon Bedrock and the AWS CDK deployed the single-shot design as a CDK application: an Amazon Simple Storage Service (Amazon S3) corpus, an Amazon OpenSearch Serverless collection and index, a knowledge base, and a data source, provisioned as CDK stacks whose order you sequenced by hand. Three things have changed since it was published. Amazon Bedrock now offers a managed knowledge base (Type: MANAGED), which owns the ingestion, storage, indexing, and retrieval itself. The OpenSearch Serverless collection and index that post provisions are no longer something you need to create, scale, or pay for with OpenSearch Compute Units (OCUs). Amazon Bedrock AgentCore provides the agentic half , a Gateway that exposes each knowledge base as a Model Context Protocol (MCP) tool, a Runtime that hosts the reasoning agent, and AgenticRetrieveStream, which reranks internally and returns a synthesized, cited answer. Agentic retrieval is available only on the Managed Knowledge Base. The observability and evaluation story is now something you can provision , not something you bolt on. This solution ships two Amazon CloudWatch dashboards spanning seven layers of telemetry, plus two forms of evaluation, on-demand and continuous, from the same deployment. This post builds that system: an agent that routes across two knowledge bases, retrieves through the Gateway, and returns a grounded, cited answer, with every layer of its behavior instrumented, as a single AWS CDK application in Python. One command deploys all of it, and CDK derives the order. If you read the 2024 post , the path from an Amazon S3 bucket to the knowledge base will look familiar, and everything after it is new: the vector store is gone, the retrieval is agentic, and roughly half of what the app provisions exists to tell you what the agent did. If you didn’t, this post stands on its own. The services at play and the path one question takes through them: the agent decides whether to retrieve and which knowledge base to consult, the Gateway exposes each knowledge base as a tool it can call, the Managed Knowledge Bases search and cite, and the loop repeats as many times as the question needs. Everything the agent does along the way lands in Amazon CloudWatch. Figure 1: The services involved and the path a single question takes through them Background Before the walkthrough, it helps to establish three things: what makes retrieval agentic, why the Managed Knowledge Base is the right foundation for it, and why observability and evaluation belong in the design rather than bolted on later. From RAG to enterprise agentic retrieval Classic Retrieval Augmented Generation (RAG) does one retrieval and one generation. Enterprise agentic retrieval puts a reasoning agent in the loop: it decides whether and what to retrieve, can retrieve several times to refine, chooses which knowledge base is relevant (semantic routing), and only then composes a grounded answer with citations. With the Managed Knowledge Base, this is a first-class service capability, the AgenticRetrieveStream API performs the multi-turn planning, executes the retrieval, and returns a cited answer, rather than an orchestration layer you build and maintain over a raw vector search. That produces better answers on complex questions, but it also produces a more complex system to operate, which is exactly why observability and evaluation are built in from the start. Managed compared to do-it-yourself Knowledge Bases Amazon Bedrock now offers a Managed Knowledge Base (Type: MANAGED): Amazon Bedrock manages the ingestion, storage, indexing, and retrieval for you, including embedding and reranking with service-managed models by default, so there is no vector database to provision, scale, or patch . This differs from a Customer-managed (DIY) Knowledge Base, where you choose and operate the vector store (for example, Amazon OpenSearch Serverless) yourself. Two differences matter most for this solution: Capability Amazon Bedrock managed Customer-managed (DIY) Agentic retrieval (AgenticRetrieveStream) Supported Not supported AgentCore Gateway integration Supported Not supported Data store Auto scaling, fully managed by Amazon Bedrock You provision, scale, and maintain it Embedding + reranking Built-in managed models, no extra cost You configure them Infrastructure to manage None Vector DB + more Because agentic retrieval and the AgentCore Gateway connector are available only on the managed knowledge base, it is the natural foundation for enterprise agentic retrieval. It also means Amazon Bedrock manages the storage and indexing for you, so there is no OpenSearch Serverless collection or OCU capacity to provision in this design. The customer-managed option remains the right choice when you want to select and operate your own vector store, for example to reuse an existing OpenSearch Serverless collection or to meet specific data-residency, tuning, or compliance requirements. For enterprise agentic retrieval, the managed knowledge base is what makes the capability possible. Why observability and evaluation An agentic system that returns an answer isn’t enough for production. You need to see how it behaves (latency, call volume, token usage), how good the retrieval and answers are, and you need those signals continuously. This solution ships two CloudWatch dashboards spanning seven layers of telemetry, plus two forms of evaluation (on-demand and continuous), all provisioned by the same CDK application. The grounding, citations, and evaluation layers also support responsible use. Cited answers trace back to their source passages so responses stay auditable, and the faithfulness and correctness scores let you catch answers drifting from their sources rather than finding out in production. Why CDK for this particular solution Any infrastructure-as-code tool c