How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks

Forum topic · 小凯 · 2026-04-28

Summary

A 2025 arXiv paper (2504.19773) by Longju Bai, Zhemin Huang, and Xingyao Wang presents the first systematic study of token consumption in AI agentic coding tasks. The authors find that agentic tasks consume roughly 1000x more tokens than code reasoning or code chat, with input tokens—not output tokens—driving the overall cost. Token usage is highly variable and inherently stochastic: identical runs of the same task can differ by up to 30x, and higher token consumption does not translate into higher accuracy. Token efficiency varies sharply across models: Kimi-K2 and Claude-Sonnet-4.5 consume on average over 1.5 million more tokens than GPT-5. Task difficulty rated by human experts correlates only weakly with actual token cost, and frontier models cannot accurately predict their own token usage, systematically underestimating real costs. These findings highlight the cost unpredictability of AI agents and the need for better cost modeling and budgeting before deployment.

Overview

Research area: NLP
Authors: Longju Bai, Zhemin Huang, Xingyao Wang
Published: 2025-04-28
arXiv: 2504.19773

Key Findings

The paper presents the first systematic study of token consumption patterns in agentic coding tasks, addressing three questions: (1) Where do AI agents spend tokens? (2) Which models are more token-efficient? (3) Can agents predict their token usage before task execution?

Agentic tasks are uniquely expensive: they consume 1000x more tokens than code reasoning and code chat, and input tokens rather than output tokens drive the overall cost.
Token usage is highly variable and inherently stochastic: runs on the same task can differ by up to 30x in total tokens, and higher token consumption does not lead to higher accuracy.
Token efficiency differs significantly across models: Kimi-K2 and Claude-Sonnet-4.5 consume on average more than 1.5 million additional tokens compared to GPT-5.
Human-perceived difficulty correlates weakly with cost: task difficulty ratings by human experts show only weak correlation with actual token cost.
Models cannot predict their own costs: frontier models fail to accurately predict their token usage and systematically underestimate actual costs.

Original Abstract (excerpt)

The wide adoption of AI agents in complex human workflows is driving rapid growth in LLM token consumption. When agents are deployed on tasks that require a significant amount of tokens, three questions naturally arise: (1) Where do AI agents spend the tokens? (2) Which models are more token-efficient? and (3) Can agents predict their token usage before task execution? In this paper, we present the first systematic study of token consumption patterns in agentic coding tasks. We find that: (1) agentic tasks are uniquely expensive, consuming 1000x more tokens than code reasoning and code chat, with input tokens rather than output tokens driving the overall cost; (2) token usage is highly variable and inherently stochastic: runs on the same task can differ by up to 30x in total tokens...

*Auto-collected on 2026-04-28.*