Can Two of Botie’s Calls Spend the Same Remaining Budget?

Botie wants to summarise a manual and compare it with the repair history. The application checks both proposals against the same remaining allowance. Each fits on its own, so both begin. Their combined work exceeds the allowance.

Checking the balance was only part of the job. We also needed to reserve the resources when admitting each call. We will build that decision, then see what changes when the user cancels, a dependency starts failing, or the next request cannot finish before the deadline.

Subtract settled usage and in-flight reservations first. Check cancellation and circuit state on a separate branch, then admit calls in the declared queue order.

{
  "accepted": [
    "manual-summary"
  ],
  "rejected": [
    {
      "id": "fault-comparison",
      "reason": "resource budget"
    }
  ],
  "remaining": {
    "tokens": 200,
    "cost_units": 4,
    "model_requests": 1,
    "tool_calls": 2
  }
}

A model request is one interaction with a provider, but an agent framework may count a model turn differently from transport attempts. A tool proposal is a request to use a capability; a tool execution is the actual work performed by its handler. Decide whether your operational counters count attempts, successes or both.

Token usage counts input and output processing. Money depends on prices and provider accounting, so equal token totals can have different costs. Elapsed time measures what the user waits for. These quantities cannot substitute for one another. The workshop carries separate tokens, cost_units, model_requests and tool_calls fields, with a wall-clock deadline checked separately.

Document counter semantics at the boundary that enforces them. A limit whose name is ambiguous will be hard to test.

A run can stay below its total allowance while one request exceeds the model’s context window or your application’s input-size limit. Measure the request after assembling its instructions, evidence and conversation state. Reserve room for the response according to the provider’s actual context rules.

The workshop supplies input_tokens and checks it against max_input_tokens. Its token reservation includes that input plus a chosen output allowance. This is a teaching contract, so a reservation smaller than the input is rejected as malformed. Real systems need a tokenizer or provider counting interface appropriate to the model, including any special messages and tool definitions.

A cumulative token cap and a per-request context cap protect different boundaries.

Settled usage records operations whose accounting has arrived. A reservation records work already admitted whose final usage is not known. Subtract both when deciding what remains.

If an operation finishes below its reserved bound, move actual usage to the settled ledger and release only the unused remainder. If accounting is missing, keep an unresolved entry and reconcile it. Treating missing usage as zero creates fictional capacity. Where provider behaviour can exceed an estimate, choose conservative limits and enforce output caps where supported. Application reservations are only as reliable as their bounds and the accounting contract.

available = limit - settled_usage - in_flight_reservations

Never subtract a reservation from one run and charge its result to an unrelated run. Keep the request and operation identities with both entries.

Our two calls each request 300 tokens. The shared remainder is 500. Checking them independently against that initial value admits too much work. Checking the first, reserving it, then checking the second produces a different outcome.

We use queue order as a transparent admission policy. Other schedulers may prioritise urgent work or maximise useful work under a budget, but they still need a shared atomic reservation step. In a multi-worker service this commonly belongs in a transaction or another concurrency-safe store. Our single-process dictionary calculation only demonstrates the policy.

A reservation is an execution decision made by the application. The model’s belief that a call is affordable does not reserve anything.

Each proposal has an id, input_tokens, timeout_ms and a reserve dictionary. The reserve dictionary names the same four resources as the run limits. A list preserves the queue order. A dictionary stores the remaining amount for each named resource, so tokens cannot accidentally be compared with tool calls.

The admission function validates every proposal, including one it will later refuse. It returns accepted IDs, rejected IDs with reasons, and the remaining budget. It does not perform network work or mutate the input ledger. That separation makes a test able to inspect the decision before any actual expense or effect occurs.

The caller must commit the returned reservations atomically before dispatching accepted calls.

available = {"tokens":700, "cost_units":14}
request = {"tokens":250, "cost_units":5}
remaining = dict(available)
if all(request[key] <= remaining[key] for key in remaining):
    for key in remaining:
        remaining[key] -= request[key]
print(remaining)
print(available)

{'tokens': 450, 'cost_units': 9}
{'tokens': 700, 'cost_units': 14}

Parallel calls spend resources at the same time. Their elapsed durations do not simply add, but each must respect the run’s remaining deadline. The fixture admits a call only if its full timeout fits before deadline_ms. It uses a shared now_ms snapshot because it is making one batch decision.

A live dispatcher must recheck remaining time immediately before each launch. Waiting in a queue consumes the same deadline. Downstream operations need the remaining budget rather than a new full timeout at every hop. Reserve time for combining results and returning a response, especially if the calls are only an intermediate stage.

The timeout is a bound on local waiting, not proof that a remote effect will be undone when the clock expires.

When the user cancels, stop admitting proposals and signal cancellation to supported in-flight operations. Keep each in-flight reservation until the operation’s final usage or an authoritative cancellation outcome is known. A closed local stream may still leave provider work running or an external write committed.

The workshop’s cancelled flag blocks the batch and leaves its supplied reservations in place. It does not simulate a remote cancellation protocol. A production UI should distinguish requested cancellation, confirmed cancellation, and an unresolved effect. Partial output also needs a clear status so it is not mistaken for a completed answer.

Returning early to the user and stopping all downstream work are separate events.

A circuit breaker watches failures associated with a dependency. In its closed state, calls can proceed subject to other limits. When the configured failure policy opens the circuit, new calls fail promptly rather than repeatedly spending the request budget on the same unavailable service.

After a recovery interval, a half-open state can allow a limited probe. A successful probe can support closing the circuit; another failure can reopen it. Our fixture receives the current state and an application-granted probe slot. It admits at most one proposal while half-open. It does not compute failure windows or recovery transitions, which need service-specific policy and tests.

An open circuit should describe the dependency that is failing. It should not silently turn every unrelated tool off.

Pydantic AI documents usage limits for requests, tool calls, tokens and cost. Its current tool_calls_limit counts successful tool invocations. Its request_limit counts model responses acted on by the agent loop, rather than every provider transport attempt. Provider retries and some continuation or fallback requests can therefore make transport activity differ from the turn count.

The enforcement moment matters too. Output-token and cost checks can occur after a response, and cancelled-stream usage may be incomplete. These controls do not all provide a pre-dispatch spending guarantee. Read the installed version’s documentation and test the cases your application relies on. Add application reservations or provider-side limits where the requirement calls for them. Keep unknown prices or usage visible as unknown.

The workshop’s queue policy is our own teaching design. It does not claim to reproduce a particular framework’s scheduling policy.

A per-run allowance protects one request. It does not stop thousands of permitted requests from exhausting a shared provider quota. A service may also need per-user rate limits, organisation spend controls, concurrency limits, and a bounded queue. Decide which boundary owns each limit.

A queue is a promise of future work. Bound it and provide a clear overload response instead of accepting an unlimited backlog. Reserve capacity for essential recovery and status work where appropriate. Otherwise an exhausted task budget might prevent the very reconciliation needed to discover whether a write completed.

Fallbacks need permissions and accounting too. Switching providers or launching a helper agent should not reset the original run’s allowance.

The first fixture exposes two proposals competing for one remaining allowance. The next gives the second call enough room by changing the existing reservation; the same construction should then admit both. A cancelled run must admit neither, regardless of its apparent budget.

The half-open fixture tests a different rule: only one probe can run even when resources are plentiful. Additional cases isolate the context cap and the deadline. Keep refusal reasons visible so a learner can explain which boundary acted. When a real service refuses work, those same reasons help distinguish expected limits from faulty accounting.

The next observability lessons will follow these decisions through traces and outcomes instead of reducing every stopped run to one generic error.