Your OpenAI bill jumps 40% month over month. Your Anthropic spend triples in a week you didn't ship anything new. The invoice tells you the total — $8,400, $22,000, whatever it is — but it never tells you why. Was it more users? A bad prompt? An agent stuck in a loop? A test project someone forgot to shut down?

Most teams respond to a spend spike by staring at a line chart and guessing. That's slow, and it's usually wrong. The faster path is to stop looking at totals and start computing four specific ratios from the usage data you already have — no tracing code, no APM tooling, just billing and usage exports you can pull today. These four ratios catch the overwhelming majority of AI cost leaks: retry storms, context bloat, cache misses, and forgotten workloads.

Leak #1: Retry Storms

When an agent or pipeline hits a rate limit, a timeout, or a malformed response, the default behavior in most frameworks is to retry — sometimes immediately, sometimes with a loop that keeps calling the model until it gets a parseable answer. One well-documented case had an agent retry loop burn up to 50x the tokens of a normal successful run before someone noticed. Uber's engineering org responded to exactly this class of problem by capping individual engineer AI spend at $1,500/month — not because anyone needed that much for legitimate work, but because a handful of misbehaving loops were quietly eating budget meant for the whole team.

The invoice-level tell is a fat tail in cost-per-session. If your median session costs $0.04 but your 95th percentile costs $6, you don't have a pricing problem — you have a retry problem. One team traced a surprise $6,531 AWS Bedrock bill to a single malformed JSON parser that triggered silent retries on every call for three days.

Metric to compute — Retry Rate: (requests flagged as retried or resubmitted) ÷ (total requests). Pull this from your provider's usage logs where request IDs repeat within a short window for the same session. A healthy retry rate is under 2–3%. Above 5%, you're paying a meaningful multiplier on every failure.

Leak #2: Context Bloat

Every multi-turn conversation or agent loop re-sends its history with each new call, because LLM APIs are stateless. That's fine for a 3-turn chat. It's a slow-motion cost explosion for a 20-step agent, because step 20 pays — in full — for the same conversation history that steps 1 through 19 already paid for. A loop that starts at 500 tokens per call can be at 12,000 tokens per call by step 20, with no code change and no new feature — just accumulated context.

This leak doesn't show up as a dramatic single spike. It shows up as a slow, steady creep in your average cost per request that's easy to miss on a monthly total but obvious once you isolate the ratio.

Metric to compute — Token-per-Request Ratio: total tokens billed ÷ total requests, tracked weekly per workload. If this ratio is flat, your usage pattern is stable. If it's climbing steadily for a given agent or feature, you're accumulating context faster than you're trimming it — and nobody is capping conversation length or summarizing history.

Leak #3: Cache Misses

Prompt caching is the single highest-leverage lever most teams never check, because it's invisible unless you go looking for it. Anthropic, OpenAI, and Google all offer cached-token pricing at a fraction of standard input cost — but caching only pays off if your prompts are structured so the static portion (system prompts, tool definitions, long context) actually hits the cache.

The gap between teams that get this right and teams that don't is enormous. A cache hit rate of 7% versus 74% on the same workload isn't a rounding error — it's the difference between paying full input-token price on 93% of your calls versus 26% of them. Teams that fix cache structure report 59–90% reductions in input-token cost on high-repetition workloads like customer support bots and coding assistants, without touching the model or the feature.

Metric to compute — Cache Hit Rate: cached input tokens ÷ total input tokens, from your provider's usage export (OpenAI and Anthropic both report this per request). If you've never pulled this number, assume it's low — most teams that check it for the first time find it under 20%.

Leak #4: Forgotten Workloads

This is the leak that has nothing to do with model behavior and everything to do with organizational memory. A background agent from a project that shipped and got deprioritized. An API key issued for a hackathon prototype six months ago. A cron job someone spun up to test embeddings that's still running nightly against a vector database nobody queries anymore. None of these show up in a sprint review. All of them show up on the invoice, every single month, as a flat recurring line with no owner and no ticket.

The tell here isn't a spike — it's the opposite. It's a suspiciously flat charge from a key or project with zero corresponding product activity. Spikes get investigated. Flat lines get ignored, which is exactly why they survive for months.

Metric to compute — Dormant Workload Cost: total spend attributable to API keys or projects with no commits, no deploys, or no linked active feature in the last 30 days. If you can't answer "who owns this key and why is it still billing," that's dormant spend by definition, regardless of the dollar amount.

The 4-Metric Scorecard

Here's the full scorecard, with rough thresholds worth flagging for review:

You can compute all four from usage/billing exports without writing tracing code: OpenAI's usage dashboard exports per-request token and cache data, Anthropic's console breaks out cached vs. uncached input tokens, and Google's Vertex AI billing export includes request counts you can join against your own logs for retry detection. The manual version takes an afternoon per provider, once, to build the pivot tables — then it's a five-minute recheck each week.

Why Single-Provider Dashboards Miss the Pattern

Here's the catch: every provider's native dashboard only shows you its own slice. OpenAI's console can't tell you that your Anthropic retry rate spiked the same week, or that a Pinecone project has been silently billing for four months with no queries. If you're running five or six providers — which is normal for any team past 10 people — you'd need to build and maintain five or six separate pivot tables, refreshed weekly, just to run this scorecard manually. Most teams try that for one month and then stop, which is exactly how leaks survive a full quarter before anyone notices.

This is the specific gap AICosts.ai is built to close: it ingests usage and billing from 50+ providers and normalizes everything into one view, so retry-rate anomalies, cache hit rate, token-per-request trends, and dormant spend show up as one cross-provider scorecard instead of six separate exports you have to reconcile by hand.

Turning the Scorecard Into a Habit

A scorecard you check once isn't a system — it's a one-time audit. The teams that actually stop leaks turn this into a monthly (ideally weekly) ritual: pull the four ratios, compare against last period, and investigate anything that crossed a threshold. That's it. No new tooling required if you're willing to rebuild the exports by hand each time.

If you'd rather not rebuild pivot tables every week, AICosts.ai tracks these ratios automatically across every connected provider and can alert you the moment a retry rate, cache hit rate, or dormant-key spend crosses a threshold — catching the leak the week it starts, not the month it shows up as a line item you can't explain.

FAQ

• Why is my AI bill so high across providers if usage feels the same? It's rarely "usage." It's almost always one of these four ratios moving in the wrong direction — retries multiplying calls, context bloat inflating tokens per call, a cache hit rate that quietly dropped, or a workload nobody's watching. Compute the four metrics before assuming you need to cut features.
• What's a normal cache hit rate for AI cost? On workloads with repeated system prompts or long static context (support bots, coding assistants, RAG pipelines), 50–75% is achievable and typical for teams who've structured prompts for caching. Below 20% usually means the static and dynamic parts of the prompt aren't separated correctly.
• What retry rate for LLM API calls should trigger investigation? Above 5% sustained over a week is worth investigating; above 10% usually points to a specific broken integration, parser, or rate-limit handling bug rather than general flakiness.
• How do I find dormant AI workload cost without engineering help? Cross-reference your provider billing export (which lists spend per API key or project) against your team's active project list. Any key billing money with no matching active project or owner is dormant by definition — flag it and confirm before killing it.