LLM API Budget Caps: Circuit Breakers, Not Dashboards

An LLM API budget cap that actually caps spend is a check that runs before every request and refuses it when the remaining budget can’t cover the worst case that request could cost. Dashboards, budget alerts and usage reports all work from data recorded after the tokens were billed, so they can tell you about a runaway agent but can’t stop one.

The pattern that does stop one is borrowed from card payments. Place an authorisation hold for the most the call could cost, make the call, then capture the real amount from the response’s usage and release the rest. Add per-tenant budgets, a circuit breaker on spend velocity and a kill switch, and keep the provider’s own spend limit switched on underneath as the backstop.

Why dashboards and alerts don’t cap LLM spend

An alert tells a person that money has already been spent. By the time they act, a loop that re-sends a growing context several times a second has kept billing the whole time. Three things make the gap worse for LLM APIs than for most cloud spend:

• The data arrives late. Anthropic says usage and cost data “typically appears within 5 minutes of API request completion” in its Usage and Cost API, and the cost report has daily granularity only. Anything built on that data is minutes behind.
• Provider limits are coarse. They apply to an organisation, workspace or project for a calendar month. They don’t know about your tenants, your agent runs or your features, so a single customer can use up the whole month’s limit for everyone.
• Provider limits have moved under people’s feet. OpenAI at one point turned its monthly project budget into an alert that kept serving requests, a change that drew a long “OpenAI removed budget limits” Hacker News thread. It has since added hard spend limits for organisations and projects, and kept alerts as a separate option that never blocks traffic.

What the provider limits actually do (October 2026)

Sources: Anthropic rate limits and spend limits, OpenAI spend limits.

Set these limits anyway. They are the last line of defence if your own guard has a bug or a key leaks. But a limit that stops every customer at once on the 19th of the month is an outage, not a budget.

The pricing your cap has to model

A lot of budget code multiplies tokens by a single input price and a single output price. That was roughly right before prompt caching tiers and batch pricing. Today the same 30,000-token system prompt costs $0.006 as a cache read and $0.12 as a 1-hour cache write, on the same model.

Claude API list prices for Claude Sonnet 5.5, October 2026 (pricing):

On top of those:

• Batch API: half price on every token type.
• US-only inference (inference_geo: "us"): 1.1x on every token type, for Claude 4.6 and later.
• Fast mode on Opus models has its own, higher prices.
• Web search costs $10 per 1,000 searches, on top of the tokens the results add.
• Cache-read discounts differ by model. Opus 5.5 reads cost 0.05x the base input price and Fable 5.1 reads 0.025x, not the usual 0.1x.
• The tokenizer changed. Claude Opus 4.7 and later models produce about 30% more tokens for the same text. Token counts or characters-per-token ratios measured on older models undercount.

Three rules follow from this:

1. Settle from usage, never from an estimate. The response tells you exactly how many tokens were uncached, written to each cache and read from cache.
2. Reserve against the dearest rate that could apply. You don’t know before the call whether the cache will hit.
3. Watch spend velocity, not just totals. A cache miss on a 30K-token prefix costs 12.5 times as much as a hit on Sonnet 5.5 ($2.50 vs $0.20 per million tokens). A deploy that breaks caching can multiply your bill without a single extra request.

Here is that pricing as code. Costs are integer micro-dollars: $X per million tokens is exactly X micro-dollars per token, so nothing is lost to floating-point rounding. Save it as pricing.py:

"""What one Claude API call costs, in integer micro-dollars (1 USD = 1,000,000).

$X per million tokens is exactly X micro-dollars per token, so the arithmetic
stays in integers. Prices: Claude API list prices, October 2026. Check
https://platform.claude.com/docs/en/about-claude/pricing before reusing.
"""

# USD per million tokens
PRICES = {
    "claude-sonnet-5-5": {"input": 2.00, "write_5m": 2.50, "write_1h": 4.00, "read": 0.20, "output": 10.00},
    "claude-opus-5-5":   {"input": 4.00, "write_5m": 5.00, "write_1h": 8.00, "read": 0.20, "output": 20.00},
    "claude-haiku-4-5":  {"input": 1.00, "write_5m": 1.25, "write_1h": 2.00, "read": 0.10, "output": 5.00},
}
WEB_SEARCH_MICROS = 10_000  # $10 per 1,000 searches
BATCH = 0.5                 # Batch API: half price on every token type
US_ONLY = 1.1               # inference_geo="us"


def actual_cost(model, usage, batch=False, us_only=False):
    """Exact cost of a finished call, from the response's `usage` object."""
    p = PRICES[model]
    creation = usage.get("cache_creation") or {}
    write_1h = creation.get("ephemeral_1h_input_tokens", 0)
    write_5m = creation.get("ephemeral_5m_input_tokens",
                            usage.get("cache_creation_input_tokens", 0) - write_1h)
    tokens = (usage.get("input_tokens", 0) * p["input"]
              + write_5m * p["write_5m"]
              + write_1h * p["write_1h"]
              + usage.get("cache_read_input_tokens", 0) * p["read"]
              + usage.get("output_tokens", 0) * p["output"])
    tokens *= (BATCH if batch else 1) * (US_ONLY if us_only else 1)
    searches = (usage.get("server_tool_use") or {}).get("web_search_requests", 0)
    return round(tokens) + searches * WEB_SEARCH_MICROS


def worst_case_cost(model, input_tokens, max_tokens, max_searches=0, us_only=False):
    """The most this call can cost: every input token at the dearest input
    rate (a 1-hour cache write), and every output token up to max_tokens."""
    p = PRICES[model]
    tokens = input_tokens * p["write_1h"] + max_tokens * p["output"]
    return round(tokens * (US_ONLY if us_only else 1)) + max_searches * WEB_SEARCH_MICROS

Reserve, call, settle: a hard LLM API budget cap in Redis

The guard keeps three things per budget in Redis: what was spent, the holds for calls that are in flight, and when each hold expires. A request is admitted only if spent + held + worst case <= limit holds for every budget it touches. The check and the hold happen in one Lua script, so two concurrent requests can’t both see the same headroom. Save it as guard.py:

"""Hard LLM budget caps: reserve the worst case, call, settle the actual cost.

Every budget (per run, per tenant per day, per tenant per month, global) is
checked and held in one atomic Redis script, the same way a card network
places an authorisation hold before it captures the real amount.
"""
import uuid

# KEYS: budget keys (one per scope).  ARGV: now, res_id, amount, hold_ttl,
# then one limit per budget key, then kill-switch and breaker keys to check.
RESERVE = """
local now, res, amount, ttl = tonumber(ARGV[1]), ARGV[2], tonumber(ARGV[3]), tonumber(ARGV[4])
local n = #KEYS
for i = 5 + n, #ARGV do
  if redis.call('EXISTS', ARGV[i]) == 1 then return {0, ARGV[i]} end
end
for i, key in ipairs(KEYS) do
  -- release holds whose caller crashed before settling
  for _, dead in ipairs(redis.call('ZRANGEBYSCORE', key .. ':exp', '-inf', now)) do
    redis.call('HDEL', key .. ':holds', dead)
    redis.call('ZREM', key .. ':exp', dead)
  end
  local held = 0
  for _, v in ipairs(redis.call('HVALS', key .. ':holds')) do held = held + tonumber(v) end
  local spent = tonumber(redis.call('GET', key .. ':spent') or '0')
  if spent + held + amount > tonumber(ARGV[4 + i]) then return {0, key} end
end
for _, key in ipairs(KEYS) do
  redis.call('HSET', key .. ':holds', res, amount)
  redis.call('ZADD', key .. ':exp', now + ttl, res)
end
return {1, res}
"""

# KEYS: the same budget keys.  ARGV: res_id, actual, now, velocity key,
# window minutes, breaker threshold, breaker key, cooldown seconds.
SETTLE = """
local res, actual, now = ARGV[1], tonumber(ARGV[2]), tonumber(ARGV[3])
if redis.call('SET', 'settled:' .. res, 1, 'NX', 'EX', 86400) == false then
  return 0  -- already settled: a retry must not charge twice
end
for _, key in ipairs(KEYS) do
  redis.call('HDEL', key .. ':holds', res)
  redis.call('ZREM', key .. ':exp', res)
  redis.call('INCRBY', key .. ':spent', actual)
end
-- spend velocity: per-minute buckets, summed over the window
local minute = math.floor(now / 60)
redis.call('INCRBY', ARGV[4] .. ':' .. minute, actual)
redis.call('EXPIRE', ARGV[4] .. ':' .. minute, 3600)
local recent = 0
for m = minute - tonumber(ARGV[5]) + 1, minute do
  recent = recent + tonumber(redis.call('GET', ARGV[4] .. ':' .. m) or '0')
end
if recent > tonumber(ARGV[6]) then
  redis.call('SET', ARGV[7], 'velocity ' .. recent, 'EX', tonumber(ARGV[8]))
end
return 1
"""


class BudgetExceeded(Exception):
    pass


class BudgetGuard:
    def __init__(self, redis, clock, hold_ttl=300, window_min=5,
                 breaker_micros=3_000_000, cooldown_s=900):
        self.r, self.clock, self.hold_ttl = redis, clock, hold_ttl
        self.window_min, self.breaker_micros, self.cooldown_s = window_min, breaker_micros, cooldown_s
        self._reserve = redis.register_script(RESERVE)
        self._settle = redis.register_script(SETTLE)

    def reserve(self, budgets, amount, tenant, model):
        """budgets: {key: limit_micros}. Returns a reservation id or raises."""
        res = uuid.uuid4().hex
        stops = ["kill:global", f"kill:tenant:{tenant}", f"kill:model:{model}",
                 f"breaker:{tenant}"]
        ok, why = self._reserve(keys=list(budgets),
                                args=[self.clock(), res, amount, self.hold_ttl,
                                      *budgets.values(), *stops])
        if not ok:
            raise BudgetExceeded(why.decode() if isinstance(why, bytes) else why)
        return res

    def settle(self, budgets, res, actual, tenant, breaker_micros=None):
        self._settle(keys=list(budgets),
                     args=[res, actual, self.clock(), f"velocity:{tenant}",
                           self.window_min, breaker_micros or self.breaker_micros,
                           f"breaker:{tenant}", self.cooldown_s])

    def spent(self, key):
        return int(self.r.get(f"{key}:spent") or 0)

The details that matter, and the payments idea behind each one:

• One atomic check across every scope. The run, tenant-day, tenant-month and global budgets are checked and held together, or not at all, the way an authorisation either succeeds or doesn’t.
• Holds expire. If a worker crashes between reserve and settle, its hold drops after hold_ttl seconds instead of locking the tenant out for the rest of the day. Set the TTL above your longest request timeout.
• Settle is idempotent. A settled:<id> key means a retried settle can’t charge twice. This matters as soon as you settle from a queue with at-least-once delivery (see offsets and delivery guarantees for why duplicates happen).
• A late settle still counts. If the hold already expired, the actual cost is still added to spent. The money was spent either way.
• The breaker and the kill switch are checked in the same script. There’s no window where a request passes the budget check after the switch was flipped.

Per-tenant quotas: which budgets to check

Every request should be checked against a small hierarchy of budgets, not just one:

Set the limits from your own data, not from round numbers. Once every call is settled into this ledger you have the real distribution of cost per run and per tenant. A run budget a few times your 99th-percentile run cost stops loops without stopping legitimate long runs. Keep the global daily budget well under the provider’s monthly limit divided by the days left, so your guard trips before the provider’s does.

Decide what the user sees when a budget is exhausted. Return a specific, consistent error that says which budget ran out and when it resets, not a generic 500 (see consistent error responses).

Circuit breakers: stop spend that is too fast, not just too much

A budget answers “how much may this tenant spend today?”. A breaker answers “is this tenant spending abnormally fast right now?”. You need both, because the expensive incidents often stay within budget for hours:

• A deploy breaks prompt caching, so every request now writes the cache instead of reading it.
• An agent retries a failing tool in a loop, re-sending its whole context each time.
• A client bug or a scraper starts calling an endpoint ten times as often.

The guard’s SETTLE script adds each settled cost to a per-minute bucket, sums the last window_min minutes, and opens the breaker (a Redis key with a TTL) when the sum passes a threshold. While the key exists, every reserve for that tenant is refused. When the TTL runs out, traffic flows again and the breaker re-trips if the problem is still there. That is the half-open state, done with an expiry instead of a state machine.

Refusing everything isn’t always the right reaction to an open breaker. For a user-facing feature, consider degrading instead: switch to a cheaper model, serve a cached answer, or queue the work for the Batch API at half price. Budgets should refuse; breakers can degrade.

The kill switch

Incidents need one control that stops spend now, without a deploy, a config reload or a person finding the right console page. In the guard that’s a Redis key: kill:global, kill:tenant:<id> or kill:model:<model>. Set the key and the next reserve is refused. Delete it and traffic resumes.

Make it boring to use. Put it in your runbook and give on-call engineers access. Store the reason as the value (incident-0412) so the refusal logs explain themselves. If the guard itself is broken, revoking or disabling the API key in the provider’s console is the switch of last resort.

Wiring the guard to the Claude API

The wrapper below counts the input tokens with the token counting endpoint, which is free and has its own rate limit. It reserves the worst case, calls the model and settles the exact cost.

import anthropic

from guard import BudgetExceeded
from pricing import actual_cost, worst_case_cost

# The guard reserves for one attempt, so the SDK must not retry behind its back.
client = anthropic.Anthropic(max_retries=0)


def provider_stop(err):
    """Classify an API error. Returns a reason when the provider itself has
    stopped spending, so the caller opens its breaker instead of retrying."""
    body = getattr(err, "body", None) or {}
    error = body.get("error", body) if isinstance(body, dict) else {}
    code = (error.get("details") or {}).get("error_code") or error.get("code")
    message = error.get("message") or ""
    if code == "enforced_spend_limit_reached":              # Anthropic tier cap, 429
        return "provider tier spend cap"
    if message.startswith("You have reached your specified"):  # Anthropic own limit, 400
        return "provider spend limit"
    if code in ("organization_spend_limit_exceeded",        # OpenAI hard limits, 429
                "project_spend_limit_exceeded", "insufficient_quota"):
        return "provider spend limit"
    return None


def guarded_call(guard, budgets, tenant, request):
    model, max_tokens = request["model"], request["max_tokens"]
    count = client.messages.count_tokens(            # free, separate rate limit
        model=model, messages=request["messages"],
        **{k: request[k] for k in ("system", "tools") if k in request})
    res = guard.reserve(budgets, worst_case_cost(model, count.input_tokens, max_tokens),
                        tenant, model)
    try:
        message = client.messages.create(**request)
    except anthropic.APIStatusError as err:
        guard.settle(budgets, res, 0, tenant)        # release the hold
        if reason := provider_stop(err):
            guard.r.set(f"breaker:{tenant}", reason, ex=3600)
            raise BudgetExceeded(reason) from err
        raise
    except Exception:
        # Timeout or dropped connection: the call may still have been billed.
        guard.settle(budgets, res, worst_case_cost(model, count.input_tokens, max_tokens),
                     tenant)
        raise
    guard.settle(budgets, res, actual_cost(model, message.usage.model_dump(),
                                           us_only=message.usage.inference_geo == "us"),
                 tenant)
    return message

Three things in there are easy to get wrong:

• SDK retries happen inside the call you’re guarding. The Anthropic Python SDK (version 1.11.0 here) retries twice by default, including on HTTP 429 responses. Retries hide repeated attempts from your guard and also retry the spend-cap 429, which can’t succeed until next month. Set max_retries=0 and retry above the guard, with a fresh reservation for each attempt.
• A provider spend limit should open your breaker, not trigger a retry. provider_stop recognises Anthropic’s enforced_spend_limit_reached, Anthropic’s own-limit 400 and OpenAI’s spend-limit codes. Anything else is a normal error. The classifier was tested against the error bodies in the providers’ docs, not against a live account that had hit its limit.
• A timeout is not a free call. When the connection drops, you don’t know what was billed. The wrapper settles at the worst case, and the daily reconciliation (below) corrects the ledger later.

Two limits of this wrapper: the token counting endpoint rejects requests that include server tools such as web search, so for those, estimate the input tokens and add max_searches to the worst case. For streaming, settle from the final message_delta event, whose usage counts are cumulative.

What the guard does under load: a simulation

To check the guard does what it claims, the script below replays three failure modes against a real Redis 7.0 server, with a fake LLM that returns realistic usage objects. It doesn’t call a real API, and its costs come from the price table above.

Save it as simulate.py next to the other two files, start a throwaway Redis (docker run -p 6379:6379 redis:7), install the client with pip install redis, and run python simulate.py. It flushes the Redis database it connects to, so don’t point it at a real one.

"""Three failure modes, replayed against a real Redis with a fake LLM.

The fake LLM returns realistic `usage` objects; nothing calls a real API.
A logical clock keeps every run identical.
"""
import random

import redis

from guard import BudgetExceeded, BudgetGuard
from pricing import actual_cost, worst_case_cost

r = redis.Redis()
r.flushdb()
now = [0]
guard = BudgetGuard(r, clock=lambda: now[0])
rng = random.Random(7)
MODEL = "claude-sonnet-5-5"
usd = lambda micros: f"${micros / 1e6:,.2f}"


def fake_llm(input_tokens, max_tokens, cache_read=0, cache_write=0):
    return {"input_tokens": input_tokens,
            "cache_read_input_tokens": cache_read,
            "cache_creation_input_tokens": cache_write,
            "output_tokens": rng.randint(max_tokens // 2, max_tokens)}


def budgets(tenant, run=None, day_limit=20_000_000):
    b = {f"budget:{tenant}:day:2026-10-05": day_limit,
         f"budget:{tenant}:month:2026-10": 200_000_000,
         "budget:global:day:2026-10-05": 500_000_000}
    if run:
        b[f"budget:run:{run}"] = 2_000_000  # $2 per agent run
    return b


# 1. An agent whose tool keeps failing: it never stops, and every step
#    re-sends the whole conversation plus the new error output.
print("1. Runaway agent loop, $2 run budget")
b = budgets("acme", run="run-1")
context, step = 8_000, 0
while True:
    try:
        res = guard.reserve(b, worst_case_cost(MODEL, context, 2_000), "acme", MODEL)
    except BudgetExceeded as e:
        print(f"   stopped before step {step + 1}: {e}")
        break
    step += 1
    guard.settle(b, res, actual_cost(MODEL, fake_llm(context, 2_000)), "acme")
    context += 4_000
print(f"   {step} steps, spent {usd(guard.spent('budget:run:run-1'))}, "
      f"next step would have sent {context:,} input tokens\n")


# 2. Eight runaway agents at once against a $5 daily tenant budget.
#    Naive: check `spent < limit`, call, then add the cost.
#    Guarded: reserve the worst case first, settle afterwards.
#    (The breaker is raised here so only the budget is being tested.)
def fan_out(guarded, tenant, workers=8, limit=5_000_000):
    b = budgets(tenant, day_limit=limit)
    day = f"budget:{tenant}:day:2026-10-05"
    naive_spent, contexts, calls = 0, [8_000] * workers, 0
    active = set(range(workers))
    while active:
        in_flight = []
        for w in sorted(active):                      # every worker checks...
            if guarded:
                try:
                    in_flight.append((w, guard.reserve(
                        b, worst_case_cost(MODEL, contexts[w], 2_000), tenant, MODEL)))
                except BudgetExceeded:
                    active.discard(w)
            elif naive_spent < limit:
                in_flight.append((w, None))
            else:
                active.discard(w)
        for w, res in in_flight:                      # ...then the calls land
            cost = actual_cost(MODEL, fake_llm(contexts[w], 2_000))
            calls += 1
            if guarded:
                guard.settle(b, res, cost, tenant, breaker_micros=50_000_000)
            else:
                naive_spent += cost
            contexts[w] += 4_000
    spent = guard.spent(day) if guarded else naive_spent
    return calls, spent, limit


print("2. Eight concurrent runaway agents, $5 daily tenant budget")
for name, guarded, tenant in [("check-then-call", False, "beta"),
                              ("reserve-then-settle", True, "gamma")]:
    calls, spent, limit = fan_out(guarded, tenant)
    over = spent - limit
    print(f"   {name:<20} {calls:>3} calls  spent {usd(spent)}  "
          f"{'over by ' + usd(over) if over > 0 else 'under by ' + usd(-over)}")
print()


# 3. A support bot with a 30K-token cached system prompt, 20 requests a
#    minute. At minute 15 a deploy puts a timestamp at the top of the
#    prompt, so every request writes the cache instead of reading it.
print("3. Cache-busting deploy at minute 15, breaker at $3 per 5 minutes")
b = budgets("delta", day_limit=100_000_000)
tripped_at, rejected = None, 0
for minute in range(30):
    spent_this_minute = 0
    for i in range(20):
        now[0] = minute * 60 + i * 3
        busted = minute >= 15
        try:
            res = guard.reserve(b, worst_case_cost(MODEL, 30_500, 1_000), "delta", MODEL)
        except BudgetExceeded as e:
            rejected += 1
            tripped_at = tripped_at or (minute, i, str(e))
            continue
        usage = fake_llm(500, 600, cache_read=0 if busted else 30_000,
                         cache_write=30_000 if busted else 0)
        cost = actual_cost(MODEL, usage)
        spent_this_minute += cost
        guard.settle(b, res, cost, "delta")
    if minute in (0, 14, 15, 16, 17, 18):
        print(f"   minute {minute:>2}: {usd(spent_this_minute)}")
m, i, why = tripped_at
print(f"   breaker opened at minute {m}, request {i + 1} ({why}); "
      f"{rejected} requests refused for the rest of the run\n")


# 4. Kill switch: one key, no deploy, takes effect on the next reserve.
print("4. Kill switch")
r.set("kill:model:claude-opus-5-5", "incident-0412")
try:
    guard.reserve(budgets("acme"), 1_000, "acme", "claude-opus-5-5")
except BudgetExceeded as e:
    print(f"   refused: {e}")

Running it with Python 3.11 and redis-py 8.1 prints:

1. Runaway agent loop, $2 run budget
   stopped before step 19: budget:run:run-1
   18 steps, spent $1.76, next step would have sent 80,000 input tokens

2. Eight concurrent runaway agents, $5 daily tenant budget
   check-then-call       80 calls  spent $5.33  over by $0.33
   reserve-then-settle   75 calls  spent $4.88  under by $0.12

3. Cache-busting deploy at minute 15, breaker at $3 per 5 minutes
   minute  0: $0.23
   minute 14: $0.23
   minute 15: $1.61
   minute 16: $0.72
   minute 17: $0.00
   minute 18: $0.00
   breaker opened at minute 16, request 10 (breaker:delta); 271 requests refused for the rest of the run

4. Kill switch
   refused: kill:model:claude-opus-5-5

What this shows:

• The run budget is the cheapest stop for a looping agent. Each step re-sends the conversation, so each step costs more than the last, and total cost grows with the square of the step count. The $2 run budget stopped the loop after 18 steps. Without it, the next step alone would have sent 80,000 input tokens.
• Check-then-call overshoots under concurrency. All eight agents passed the spent < limit check before any of them was charged, so the last round went $0.33 over a $5 budget. The overshoot grows with the number of calls in flight times the cost per call: eight agents here, but it could be hundreds of concurrent requests for one tenant in production. Reserve-then-settle never went over. It finished $0.12 under the limit, because the next round’s worst-case holds didn’t fit.
• The breaker caught what the budgets couldn’t. At $1.61 a minute, a $100 daily budget would have absorbed the cache-busting deploy for about an hour. The breaker opened about 90 seconds after the deploy, on the 10th request of minute 16.

Which control stops what

A rule of thumb for where the guard should live:

• One service calls the LLM: put the guard in that service, as above.
• Several services or teams share a provider account: put it in a gateway they all go through. Before you rely on it, check whether the gateway reserves before the call or only counts spend afterwards. If it only counts afterwards, it overshoots under concurrency in the same way as check-then-call.
• Either way: set the provider spend limit too, below the amount that would hurt.

Failure modes and their symptoms

• The spend-cap 429 is treated as a rate limit. Symptom: every request returns 429 without a retry-after header, and retries never succeed. Fix: read error.details.error_code (Anthropic) or error.code (OpenAI) and open the breaker until the reset date.
• Leaked holds. Symptom: a tenant is refused while spent is well under its limit. Fix: expiring holds, as in the script, and alert on the number of active holds per tenant.
• Double-charged settles. Symptom: your ledger shows more spend than the provider’s cost report. Fix: idempotent settle keyed by reservation ID.
• Worst-case holds starve concurrency. Symptom: refusals while there is still budget left, as in the simulation’s last round. Fix: lower max_tokens to what the feature needs, and if you never use the 1-hour cache, reserve input at the 5-minute write rate instead.
• Batch jobs settle much later. Batch results can take up to 24 hours. Reserve at submission time, at batch prices, with a hold TTL longer than the batch can run.
• Period boundaries. Anthropic’s tier cap resets at 00:00 UTC on the 1st. If your tenant budgets reset on local midnight, write down which clock each budget uses.
• Redis is down. Decide in advance whether the guard fails open (keep serving and rely on the provider limit) or fails closed (refuse). A reasonable split: fail closed for free tiers and batch work, fail open with a tight provider limit for paying customers.
• Stale prices. Prices change and new models arrive with new rates. Keep the price table in config with the date you checked it, and fail closed on a model the table doesn’t know.

Reconcile your ledger like a payments team

A guard that settles every call is also a ledger: who spent what, on which model, for which feature. Reconcile it once a day against the provider’s own numbers. On the Claude API, the Usage API reports tokens in 1-minute, 1-hour or 1-day buckets, filterable by API key and workspace. The Cost API reports daily costs in USD, grouped by workspace. Both need an Admin API key.

If the two disagree by more than a small percentage, something is wrong. Usual causes are a model missing from your price table, a code path that bypasses the guard, or timeouts settled at the worst case. That’s the payments habit at the centre of this post: your internal ledger is the control, and the provider’s invoice is how you audit it.

For the cost side of the same coin, how much caching changes what each request costs, see Agent memory vs RAG vs long context, which runs the numbers with the same price table.

FAQ

Does OpenAI have a hard spend limit?

Yes, again. After a period when project budgets only sent alerts, OpenAI added hard spend limits for organisations and projects. Requests over the limit fail with HTTP 429 and organization_spend_limit_exceeded or project_spend_limit_exceeded. OpenAI notes that enforcement isn’t instantaneous, so recorded spend can slightly exceed the limit.

What happens when I hit the Claude API spend limit?

If it’s your tier’s cap, requests return HTTP 429 with enforced_spend_limit_reached and no retry-after until 00:00 UTC on the 1st of next month, unless you move to a higher tier. If it’s a limit you set yourself, requests return HTTP 400 invalid_request_error until you raise the limit or the month ends.

Does max_tokens change what I pay?

No. You’re billed for the output tokens actually generated. max_tokens sets the worst case, so it decides how much budget each call has to reserve. Setting it to what the feature really needs lets more calls run at once within the same budget.

Should the budget check live in a gateway or in the application?

In whichever layer every request already passes through. The application knows about agent runs and features, so it can enforce per-run budgets. A gateway is the only place that sees all traffic when several services share one account. Many teams use both: the gateway enforces per-team and per-key limits, and the application enforces per-run limits.

Isn’t reserving the worst case wasteful?

A hold isn’t a charge. It only limits how many calls can be in flight near the end of a budget. In the simulation, reserve-then-settle used all but $0.12 of a $5 budget, which is less than the cost of one more call. The alternative, overshooting by however many calls are in flight, is what a cap exists to prevent.

Bottom line

Treat LLM spend the way a payment system treats money: authorise before you act, capture what was actually used, and reconcile against the provider’s invoice every day.

1. Reserve the worst case against run, tenant and global budgets in one atomic step, and settle from usage.
2. Add a spend-velocity breaker for incidents that stay within budget, and a kill switch for everything else.
3. Keep the provider’s spend limit on underneath, as the backstop.

Dashboards still have a job: telling you where the money went. They just can’t be the thing that stops it.