Architecture patterns for fast session startup on Amazon Bedrock AgentCore runtime

Agents today review code, process claims, and run unattended for hours. Many also sit behind interactive experiences where a person waits for the first response. Amazon Bedrock AgentCore runtime gives these agents a fully managed, serverless environment: each session runs in its own isolated microVM, the platform scales to zero when there’s no work, and you only pay for the resources your sessions consume. AgentCore runtime with platformVersion set to “V2” changes how sessions start. Instead of initializing your container for every session, it prepares your environment once, snapshots it, and restores that snapshot for each new session. Session start times are faster and more consistent. V2 also reclaims memory your sessions don’t use, so your bill tracks what your agent uses. For most agents, switching to V2 is all it takes. Some agents need restructuring to get the full benefit, and for chat and voice experiences even 2 seconds is too slow. This post explains how V2 starts a session, how to measure session start time, and how to keep work off the first request, including build hooks, which AgentCore runtime now supports. We cover four patterns, from changes to your agent’s code to starting sessions before users need them. Which pattern do you need? Your situation What to do Section A new session in about 2 seconds is fine Switch to V2 and do all initialization before app.run() Keep work off the first request Your agent needs large files, a CLI, or a browser early, and can’t hold them in its process Add a build hook to load and prime them while the snapshot is built Working with hooks You need sub-second, and users engage before they send their first request (opening the app or a chat window) Start the session in the background when the user engages Pre-warm on user engagement How V2 starts a session AgentCore runtime runs each session in a new microVM. On V1, AgentCore pulls your container image from Amazon Elastic Container Registry (Amazon ECR), starts the container, and runs your initialization code before it can process the first request. The image pull dominates, so cold start grows with image size. V2 does that work once. When you create or update a runtime, AgentCore runtime starts your container in a fresh microVM and waits for the first healthy ping ( GET /ping ), which must arrive within 120 seconds. It then clears the operating system’s file cache, to keep the snapshot small, freezes the microVM, and stores its memory and disk as a snapshot. This is a change from V1: Every create or update now takes a few minutes while the snapshot is built. In this post, init means everything your agent does before /ping reports healthy. With the AgentCore Python SDK, that’s everything before app.run() . If you run your own HTTP server, in Node.js or another language, report healthy only after your initialization finishes. Figure 1: Build, once per create or update When a new session arrives, AgentCore restores the snapshot into a new microVM. Your process resumes exactly where it was frozen, with its imports, objects, and clients in place. Files from your image are loaded lazily: each block is fetched the first time a session reads it. Figure 2: Invoke, every new session Three facts follow from these flows, and the rest of this post builds on them: Memory comes back whole. Anything your process held at snapshot time, and anything in a RAM-backed file system such as /dev/shm , is available at memory speed from the first access. File reads don’t carry over. The file cache is cleared before the freeze, so reading a file during init doesn’t make it fast in a restored session. Image files are fetched on first read. The first read of a block comes from snapshot storage and is then cached closer to the host. The following table shows what that means for the work a typical agent does, depending on where it does it: Work Done during init Done in the handler Import libraries In memory when the session starts Read from the image in every new session Create clients (for example, boto3 ) Restored with endpoint and credential resolution done. Sockets reconnect on first use Built from scratch in every new session Open network connections Stale after the restore. Close them before the snapshot Opened fresh Load static data from the image and keep it in an object In memory when the session starts Read from the image in every new session Read a file from the image without keeping the data No effect, because the file cache is cleared Read from the image in every new session If you know AWS Lambda SnapStart, this will feel familiar. Marc Brooker’s post on how SnapStart loads snapshots and the SnapStart priming strategies post are useful background. Everything in the table assumes your agent can keep what it needs in its own process. That isn’t always possible. A CLI that runs as a separate process and exits, or a browser binary, is read during init and then gone, so a restored session fetches it from the image again. For these cases, you can add a build hook, covered in “Working with hooks.” Figure 3 shows where it runs. Figure 3: Build and invoke with hooks Measure before you optimize Latency in an agentic application isn’t one number. A request spends time in AgentCore runtime getting a session ready to run your code, then in your code: model calls, tool calls, sub-agents, and streaming the response back. You fix these in different ways, so start by finding out which one dominates. AgentCore runtime publishes both as Amazon CloudWatch metrics in the AWS/Bedrock-AgentCore namespace. Latency is the time to first byte. OverheadLatency is the part that belongs to the platform, such as starting the microVM and restoring the snapshot on a new session. What you want How to get it TTFB (Time to first byte) Latency Runtime time OverheadLatency Your code’s time Latency − OverheadLatency Init time Measured in your code at startup (see the following example). It isn’t part of any request on V2 Init doesn’t appear in these metrics because on V2 it runs once, while the snapshot is built, and not in each session. It adds to how long a create or update takes to reach READY, not to request latency. To see how long it takes, log it once before app.run() . The line appears in your runtime’s build logs: import time INIT_START = time.time() # first line of your module # imports, clients, data loading ... print(f"init_ms={round((time.time() - INIT_START) * 1000)}") app.run() How to read the split: High OverheadLatency on new sessions: the time is in session start. Use the patterns later in this post to start sessions before users need them. Your code’s time is much higher on the first request than on the second in the same session: work that belongs in init is running in your handler. Add custom spans in AgentCore Observability (a capability of Amazon Bedrock AgentCore) around tool setup, file loads, and late imports to find it, then move it into init. Report percentiles such as P90 and P99, or trimmed means such as TM90, which averages the fastest 90% of requests and drops the slowest tail. A plain average hides the slow requests. OverheadLatency covers requests to running sessions as well as new ones, so its distribution depends on how many of your requests start a new session. Compare runs with a similar mix. Test at the rate of new sessions you expect in production, not with a handful of sessions after a deploy. To include network time from your client, measure on the client as well. Time the invoke call, have your handler return how long it ran, and subtract. Both timestamps come from the same machine, so clock differences between the client and the microVM don’t matter. If you stream, very small events can be held back until a few hundred bytes of output build up, so time to first event can read higher than session start. Keep work off the first request The usual cold start advice is to import lazily and keep initialization small. On V2, do the opposite: init runs once, when the snap