Run the same agent task twice and the API bill can differ by how much? Answer: 30x. This is not clickbait — it's cited from Bai et al. 2026 in a new paper: different executions of the same task can differ in token consumption by more than an order of magnitude.
The cause is hidden in how agents work. Every step an agent takes may trigger a new model call, and the context left by earlier steps becomes the input to every subsequent call. Generate one extra paragraph early on, and each of the next ten calls carries that extra paragraph as input. Context isn't a one-time expense — it's compound-interest debt. Layer on the model's stochastic behavior — same input can take different paths, retry different numbers of rounds — and the bill is unpredictable before the run even starts.
TokenCast (arXiv 2609.35760), from Sun Yat-sen University, RPI, Southeast University, and HKUST, exists to predict this debt: 11,712 execution traces, 240 tasks, 6 agent models, 4 benchmarks (SWE-bench Verified, Search-R1, MMLU-Pro, LongBench-v2), code open-sourced.
Core mechanism: from addition to composition
Why do traditional predictors fail? Because they sum per-step costs as independent items. TokenCast instead records three numbers per execution segment: how much it spent itself, how much context it added, and how much the starting point shifted. The most critical term in the composition formula is the third — every later segment must reread the context introduced by earlier segments; that's the rereading tax. One-sentence summary: the context you owe early on gets paid back with interest on every subsequent call.
The predictor itself is surprisingly plain: LightGBM. 0.8 ms per inference, 32.8 ms cumulative for the full pipeline (feature extraction plus composition) per task, zero API calls. Using a tree model to predict an LLM's spending is itself worth pondering: how an agent spends money is learnable.
Fact-checking the numbers first
Every number in the source material was checked against the paper: 30x ✓ (cited from Bai et al., attributed), compositional cost representation ✓, zero extra LLM calls ✓, 32.8 ms ✓, 14.5% MAE reduction (averaged over 96 combinations) ✓, 21.3% savings ✓. The numbers all check out — this secondary coverage is faithful. But there are three instances of selective emphasis, all printed in the paper's own tables:
First, the 14.5% is selectively read. The paper breaks results down by four prediction points: at task start (Task Start), an average of −2.2% — losing to the strongest baseline; at each call start, +1.9%, roughly a tie; rolling updates during generation, +30.4%; mid-task updates, +27.8%. The 14.5% average is entirely propped up by the latter two. In other words: it doesn't win at ex-ante prediction, it wins big at in-execution correction. And the people paying the bills most want ex-ante prediction. As noted in prior hardware-event research: "benchmarks are assertions, daily use is execution" — this is the isomorphic cost version: losing in the benchmark cell, winning in the daily-use cell.
Second, the 21.3% has two qualifiers. One: it's offline replay — 288 GPT-5.4 traces, with stopping-rule experiments across seven budget tiers, not live real-world intervention. Two: the paper itself states in a footnote that "replay does not test whether the task was actually solved" — so the claim of "no sacrifice in task completion rate" refers to completion in the sense of reaching the recorded endpoint, not getting the problem right. Actually wiring these predictions into a runtime for real-time intervention is, in the paper's own words, future work.
Third, the 32.8 ms caliber. It's the cumulative time for predicting one full task, not single-response latency. "Millisecond-level" is fair, but it's not "fast at every step" — it's "cheap for the whole run."
Tying into the broader narrative
Three threads connect here. First, the interface-tax ledger previously covered the KV-cache tyranny — switching models means switching interfaces, with prefix taxes paid anew; TokenCast quantifies the within-task compounding of prefix tax — the same physical intuition, static vs. dynamic versions. Second, the ripwire case established "request-level budget parameters" as the fourth cell of interface-tax pricing — ask for rather than discover; TokenCast effectively installs a dashboard on that parameter, so you can finally ask. Third, verification-bandwidth economics: the prediction itself costs 32.8 ms and zero tokens — this nearly free ledger makes runaway costs foreseeable. The cheapest form of control is knowing in advance which call will go off the rails.
A falsifiable prediction: within twelve months, mainstream agent frameworks will build in cost prediction and stopping rules, exposed as request parameters akin to max_estimated_cost — if that doesn't happen, it means cost unpredictability isn't yet painful enough for users.
The piece ends where it began: with the 30x itself. Variance isn't noise — it's the shadow of behavioral unpredictability: one extra retry, one extra file read, and the bill doubles. A predictor can't eliminate variance, but it moves "runaway" from the unknowable portion of the bill to the portion visible before launch. The 21.3% savings is a byproduct; the real deliverable is turning budgeting from hitting a wall into navigation.
---
Sources: full paper (arXiv 2609.35760v2 — abstract, four-quadrant data, Equation 1, Table 3, LightGBM comparison, budget replay section checked paragraph by paragraph), GitHub repository (DEFENSE-SEU/TokenCast). The 30x figure is cited from Bai et al. 2026. The source material's "no sacrifice in task completion" and "automated circuit-breaking" were adjudicated as selective emphasis against the paper's own framing. Published 2026-10-06.