Optimizing cost per task and proving agent ROI on Amazon Bedrock

Part 1 of this series built the operating model that controls Amazon Bedrock spend: attribution with AWS Identity and Access Management (IAM) principal tags and Cost and Usage Report 2.0, observability with model invocation logging, enforcement at three tiers, and a review cadence. That layer answers who spent it, and you can stop them. It can’t tell you whether the spend was well spent. This post answers two questions that Part 1 deliberately left open. First: Why is this workflow expensive? The answer requires thinking in cost per task, not cost per token. Second: Is the agent better than the humans were? The answer requires baselines, controlled experiments, and a scorecard tied to decisions. In this post, you learn a framework for calculating true model cost (including four hidden cost drivers that the pricing page doesn’t show), techniques for reducing cost per task through caching, routing, and output constraints, and a discipline for measuring agent ROI with human baselines, holdout experiments, and a five-dimension scorecard. This post is written for two audiences: the application and platform engineers who own the optimization levers, and the product owners and FinOps partners who need to justify agent spend to leadership. Cost per task, not cost per token Price per token invites quick comparison, but it doesn’t reflect what you actually pay. The published rate reflects the cost of a single token. Your bill reflects the cost of completing a task. A task is one complete unit of work your application asks a model to do: resolve one support email, extract fields from one invoice, or answer one user question. A task runs from the request through to a usable result, however many model calls it takes. The lower-priced model might not be the cheaper one to run. A cheaper model might fragment your text into more tokens, pad responses with filler, misread your tool schema and loop through corrections, or burn thousands of reasoning tokens before answering. None of these costs appear on the Bedrock pricing page . The four hidden cost drivers Four model characteristics determine what that published rate actually costs you on your workload. Tokenizer efficiency A token isn’t a word, and it is not equal across models. Every model family uses a different tokenizer and vocabulary. The same text, code, or JSON payload splits into different token counts. If Model A’s list price is 15 percent lower but its tokenizer needs 40 percent more tokens for your data (common with structured schemas and numerical arrays), Model B is cheaper. The gap widens for non-English text: many tokenizers fragment non-Latin scripts into two to three times more tokens than the equivalent English. Output alignment and verbosity Output tokens commonly cost 3–5x more than input tokens (the ratio varies by family. Refer to the Bedrock pricing page ). A model that ignores formatting constraints like “return only raw JSON” makes you pay that output premium for filler words. A model that follows formatting instructions closely saves money by generating fewer of these more expensive output tokens. Agentic loop efficiency Agentic workflows change the cost math entirely. Tool use (function calling) lets a model invoke external functions, requiring a multi-turn loop: analyze the prompt, decide to call a tool, wait for the observation, generate an answer. Each turn re-sends the full conversation history and tool results as input tokens. A model that takes four turns (each one re-sending a larger context payload) costs far more than one that finishes in a single pass. Reasoning overhead Many models support extended thinking , working through a chain of thought before the visible answer. Reasoning tokens bill as output tokens, so a model that thinks for 2,000 tokens before a 200-token answer bills you for 2,200. Some models redact the reasoning while still charging for it. Start with reasoning turned off and enable it only when benchmarks show a quality gap that justifies the cost. Cost per task: A worked example Figure 1: Comparing two models across five cost dimensions. List price is only one of five factors that determine actual cost Consider an agentic workflow that reads a 750-word customer email, extracts entities, calls an API to verify account status, and outputs strict JSON. The following comparison uses illustrative prices (always benchmark against your own data and current Bedrock pricing ): Metric Model A (lower list price) Model B (higher list price) Input list price $0.50 / 1M tokens $1.00 / 1M tokens Output list price $1.50 / 1M tokens $3.00 / 1M tokens Tokenizer efficiency (turn 1 input) 1,500 tokens 1,000 tokens Output verbosity (final response) 450 tokens 150 tokens Agentic loops required 3 turns (error correction) 1 turn (zero-shot) Cumulative input tokens, all turns 6,150 1,000 Cumulative output tokens, all turns 750 150 Total task cost $0.0042 $0.00145 Model A: (6,150 x $0.50 + 750 x $1.50) / 1,000,000 = $0.0042 Model B: (1,000 x $1.00 + 150 x $3.00) / 1,000,000 = $0.00145 Figure 2: Why the higher-priced model costs 65% less. Each extra turn re-sends a growing context payload, compounding the bill Why the 2x-priced model costs 65 percent less: three compounding factors. Model A’s tokenizer fragments the email into 50 percent more tokens, inflating every input payload. It pads its JSON with filler, tripling output tokens. And it misaligns with the tool schema, needing two correction turns that each re-send a growing history. Cost per successful task The preceding example assumes both models eventually succeed. In production they won’t, and a failure is rarely free. The formulas: For retry-until-success workflows: cost per successful task = cost per attempt / success rate For escalate-on-failure workflows: cost per successful task = cost per attempt + (1 - success rate) x escalation cost Input further splits into three buckets when prompt caching is active: uncached input at base rate, cache reads below base rate, and cache writes above base rate. Measuring it on your own workload Which model is actually cheaper only shows up when you benchmark against your own data. Amazon Bedrock model evaluation is the managed feature for the quality half of that: define criteria, upload your prompt datasets, and get scored results with automatic metrics and optional human review, with no custom infrastructure required. Quality scores alone don’t produce a bill, though. Three more Bedrock features supply the token and turn counts you price against them: CountTokens returns the input token count a given model would charge for a given payload without running inference. It’s the cheapest way to compare tokenizer efficiency (driver #1) across candidates before you spend anything on generation. Model invocation response carries a usage block with inputTokens , outputTokens , cacheReadInputTokens , and cacheWriteInputTokens . Accumulating those across the turns of a single task gives you the four terms of the preceding cost-per-attempt formula. One gotcha: when caching is enabled, inputTokens counts only the uncached tokens, so your total input volume is inputTokens + cacheReadInputTokens + cacheWriteInputTokens . Sum them to know the footprint, but price them separately, since the three bill at different rates. Evaluate this multiple times throughout your journey and set the cache checkpoints based on what is the active current load. The drift in cache checkpoints happens as the application progresses and the tools become stale. This means that you will be writing more to the cache in 3 months from now when the drift has happened. Evaluating your data constantly for optimal cache usage becomes a primary lever to reduce cost. Model invocation logging writes per-invocation token counts to Amazon Simple Storage Service (Amazon S3) or Amazon CloudWatch Logs, so you can measure turns-to-completion against real production traffic instead of a sample. Tag calls with request metadata to slice those lo