Groundhog Day for AI agents: Prompt caching patterns for using OpenAI models on Amazon Bedrock

Every time you spin off your agent, it has to re-read its whole life story before performing an action. Here’s how to make it remember without having to feed it the same context again. In the movie Groundhog Day , Bill Murray’s character re-lives the same February 2 every day. He wakes up to the same song playing on the radio, in the same inn, and the same small town, again and again for what some movie fans estimate to be 34 years. Nothing he accomplished the day before carries over unless he works it out again from scratch. AI agents live in a version of Groundhog Day. Every time an agent takes a step (calls a tool, reads the result, decides what to do next), it has to send the model its entire story so far to maintain continuity: the tool definitions, the instructions, every previous tool call, and every result. The model only predicts the next token based on the input you provide. It has no internal memory between requests, so it must read all of it again. By turn 20 of a coding session, the model re-reads turns 1–19, and it’s you who pays for every one of those tokens, again and again. Prompt caching breaks the cycle. The service stores your input prompt (instructions, tools, and content) and reuses it on the next request, at a fraction of the input price. In this post, I walk you through an agent’s life, from its first turn to when it gets new tools and spawns helpers. Along the way, I show you how prompt caching keeps costs down and keeps agents responsive and consistent. What an agent sends to its LLM Before talking about caching, it helps to look at what a typical agent’s first invocation to the large language model (LLM) contains: Tool list – The definitions of every function the agent can call. Changes when you ship a new release. Instructions – The developer prompt: rules, persona, policies, examples. Also changes per release. History – Everything that happened in this session so far: the task, the model’s tool calls, and the tool results (which grows every turn). New turn – The latest tool result or user message (which might be different every time). What an agent sends to the model, turn after turn if you don’t set up caching: Figure 1: What an agent sends to the model on every turn How prompt caching works under the hood When a model reads a prompt, it first does a write called prefill . During prefill, it processes every input token and builds its key and value vectors (also known as key-value, or KV), which form the model’s working state. This happens before the model writes its first token. Prompt caching saves that state, which contains the system prompt, tools, content, history, and so on. When a later request starts with the same input tokens (that is, the same text), the prompt caching service loads the saved state instead of computing it again. It then processes only what is new in the next payload to the model. Prompt caching can be implicit, where Amazon Bedrock selects where the reusable prefix ends, or explicit, where your code marks that endpoint using prompt_cache_breakpoint . You can use both approaches together, and we recommend doing so. Turn 1 compared to turn 2 Figure 2: Turn 1 saves the prompt prefix up to the breakpoint, and turn 2 reuses it and pays full price only for the new question Implicit or explicit, and why do we need both? For agentic workloads, implicit caching lets Bedrock choose what to cache as the conversation grows. With explicit caching, you can mark shared content, such as tools and instructions, for reuse across sessions and when spinning up helper agents. You don’t always need both forms of caching, but combining them gives you a reusable starting point while Bedrock manages the growing conversation automatically. New sessions can also reuse that shared starting point while each conversation builds on its own cached history. In this post, we start with the first turn, then show how to carry cached content through later turns, new tools, and across subagents. Hello cached world The first win is to cache the static part of the prompt: the tools and instructions that stay the same across requests. These examples use your existing Amazon Bedrock client, TOOLS , and task . Replace the shortened INSTRUCTIONS string with your real policies and examples. The reusable prefix must reach 1,024 tokens to qualify. When caching static content, use explicit mode: choose where the saved prefix ends. KEY = "support-agent:v7:gpt-6-astra" INSTRUCTIONS = "Your full support policies and examples..." # Rendered context: [provider context][TOOLS][INSTRUCTIONS] | [task] # Cache starts at the beginning; the marker below chooses its end. resp = client.responses.create( model="us.openai.gpt-6-astra", store=False, # Response storage; prompt caching still works. include=["reasoning.encrypted_content"], prompt_cache_key=KEY, prompt_cache_options={"mode": "explicit", "ttl": "30m"}, tools=TOOLS, # Rendered before instructions; keep order stable. input=[ { "role": "developer", "content": [{ "type": "input_text", "text": INSTRUCTIONS, # Save through this whole block, including TOOLS. "prompt_cache_breakpoint": {"mode": "explicit"}, }], }, # User message is processed but not cached. {"role": "user", "content": task}, ], ) The prompt_cache_key tells the service which requests are meant to share a cache. We come back to how to use it when we get to reading the cache in new sessions. In one of our tests using explicit caching, the roughly 4,000-token tools-and-instructions prefix wrote 4,120 tokens on the first request and read those 4,120 tokens on later requests. The task (or user request) comes after the cache boundary, so this doesn’t trigger a write to the cache. Keep in mind that prompt caching works only when you pass exactly the same text. Make sure that the agent’s instructions and tool lists are serialized the same way every time. Model Context Protocol (MCP) servers and other forms of tool delivery need to be indexed to make sure they stay consistent. Keep anything that varies, such as the date, the user’s name, or a request ID, after the cached instructions. As the content grows and varies (different tools and content), use implicit mode to let the service follow the conversation. Here, the first request has no explicit marker. Its latest eligible message is the user’s task, so the automatic breakpoint comes after that task and covers the tools, instructions, and task together. The mode is shown explicitly in the following example for clarity, even though implicit is the default. Setting store=False leaves conversation history in your application. It doesn’t disable caching. def ask_with_implicit_cache(history): return client.responses.create( model="us.openai.gpt-6-astra", store=False, include=["reasoning.encrypted_content"], prompt_cache_key=KEY, prompt_cache_options={"mode": "implicit", "ttl": "30m"}, tools=TOOLS, input=history, # Everything gets cached ) history = [ {"role": "developer", "content": INSTRUCTIONS}, {"role": "user", "content": task}, ] # Automatic endpoint: the end of the user message. resp = ask_with_implicit_cache(history) This is useful when the next turn continues the same task. A different task doesn’t match the prefix through the old task, and the implicit write doesn’t necessarily create a separate entry for only the tools and instructions. Use an explicit marker there when you need that shorter shared prefix to be reusable across tasks. Use both when the agent needs a shared starting point and a growing history. Start with the marked developer block in the following code example and call the function ask_with_implicit_cache . The request-level mode stays implicit. The block-level marker adds an explicit save point after the instructions. The service can also save through the latest eligible message. This keeps the shared prefix available while the conversation grows. history = [ { "role": "developer", "content": [{ "type": "input_text", "text": INSTRUCTIONS, "prompt_cache_breakpoint": {"mode":