Smart AI, Clumsy Butler: When AI Agents Learn to "Size Up the Task"
> When an AI is told to "change a lightbulb," it reviews the electrical schematic for the entire building—without knowing how silly that is.
This post reviews the paper "Do AI Agents Know When a Task Is Simple? Toward Complexity-Aware Reasoning and Execution" by Junjie Yin and Xinyu Feng (arXiv:2607.13034).
The Problem: Maximum-Context-First Waste
Imagine asking an assistant to change a Gmail icon's color from blue to red—a two-minute job. Instead, it reads the whole design system, checks every page using the icon, reviews component library history, consults three unrelated teams, and performs a full brand audit. Three hours later: "Done."
This is today's AI Agent reality. They cannot judge task difficulty; they only know one strategy: read all possible context, then act. This "maximum-context-first" (MCF) strategy turns a one-line edit into a mini codebase audit.
Human engineers intuitively form an "initial operating point"—like power-flow solvers that compute a structured starting point before Newton–Raphson refinement, instead of enumerating the whole state space.
Agent Cognitive Redundancy Ratio (ACRR)
The paper defines:
ACRR = (Actual cost − Minimum sufficient cost) / Minimum sufficient cost
where minimum-sufficient execution is the lowest-cost agent trajectory that satisfies success constraints.
Shocking finding: redundancy is highest on the simplest tasks. On a trivial two-line edit, the strongest baseline hits ACRR = 10.1—over 1000% extra effort.
The E3 Framework: Estimate → Execute → Expand
The core idea, via analogy to finding a TV remote:
- Traditional AI: search every corner of the house, attic to basement.
- E3: estimate likely locations (sofa? coffee table?), check those first. Found? Done. Not found? Expand to the bedroom, then kitchen.
- 84.9% lower cost
- 90.9% fewer tokens
- 92.2% fewer files inspected
- Level 1 (simplest): 22.1
- Level 2: 11.0
- Level 3: 5.4
- Oracle cost: 6.0
- MCF: read all 7 project files, 1844 tokens, ACRR = 10.1
- Adaptive Retrieval: 1 retrieved file, ACRR = 2.27
- E3: zero irrelevant files inspected, local single-file fast path, ACRR = 0.59
- Yin, J., & Feng, X. (2026). Do AI Agents Know When a Task Is Simple? Toward Complexity-Aware Reasoning and Execution. *arXiv preprint arXiv:2607.13034*.
- Tinney, W. F., & Hart, C. E. (1967). Power flow solution by Newton's method. *IEEE Transactions on Power Apparatus and Systems*, (11), 1449-1460.
- Stott, B. (1974). Review of load-flow calculation methods. *Proceedings of the IEEE*, 62(7), 916-929.
Key insight: make a fast, cheap estimate, execute the minimal viable path, and expand scope only when verification fails. E3 is not "thinking less"—it is "judging correctly," achieving both speed and reliability without trading one for the other.
This replaces "maximum-context-first" with "progressive context expansion"—instead of always carrying a full toolbox, check the weather first and buy a screwdriver locally if needed.
Results: MSE-Bench
The authors built MSE-Bench (Minimum-Sufficient Execution Benchmark): 121 deterministic code-editing tasks generated programmatically from task prototypes, each with an oracle minimum-sufficient trajectory, across three difficulty levels (single-file, cross-file, repo-level).
| Strategy | Success rate | Avg cost | ACRR | |------|--------|----------|------| | MCF (max-context-first) | 100% | 122.9 | 12.9 | | Fixed ReAct | 66.9% | - | 1.29* | | Adaptive Retrieval | 100% | 22.1 | 1.21 | | E3 | 100% | 18.6 | 0.55 |
*Fixed ReAct's ACRR computed only on solved tasks (it failed all Level-3 tasks).
E3 at 100% success rate delivers, vs. MCF:
And a further 16.0% cost reduction vs. the strong adaptive baseline.
The Counterintuitive Finding
MCF's ACRR by difficulty:
The simpler the task, the worse the waste—because MCF's cost is fixed (always reading the whole repo) while minimum-sufficient cost shrinks. E3 keeps a flat profile: 0.64 / 0.26 / 0.73.
The Gmail Icon Case
E3 cuts cost by 85.8% vs. MCF and 51% vs. Adaptive Retrieval.
Real-World Validation
The LLM-Case framework tested three strategies on real GPT-4o against open-source repos, grading every candidate patch by running actual pytest suites. Over-reading exists on real models (though milder than simulated worst cases), and E3 remained the leanest, fastest strategy. Its only "failure" was a provider rate limit—proof of how fast it is.
Adversarial testing with reserved instruction phrasing that breaks the estimator's lexical cues preserved E3's 100% success rate—showing its advantage comes from the Estimate-Expand architecture, not memorized benchmark templates.
Engineering-Grounded AI (EGAI)
The paper's larger vision: agent effort should match the task's engineering reality and be anchored by verification. E3 is orthogonal to adaptive effort routing (which tunes "how much to think" or "which engine to use"), distinct from any-time/bounded solvers, and differs from metacognition by asking "how much information does this task need?" rather than "do I know the answer?"
Future directions include calibrated, learnable task-state estimators, richer MSE-Bench complexity mechanisms, per-phase effort routers, and extension to SWE-bench-style real-world tasks.
Conclusion: Efficiency Is a Virtue
Humans instinctively know: changing a lightbulb doesn't require reading the circuit diagram. AI Agents, however, are designed to "assume the worst for safety," wasting enormously on simple tasks. As the paper puts it:
> "True intelligence is not only the ability to solve hard problems, but also to recognize when a problem is simple—and act accordingly."
Efficiency is not spending less; it is spending correctly. The same judgment that removes waste from simple tasks can provide needed coverage on hard ones. Like a chef knowing when to sear hot or simmer slow, AI is finally learning this judgment—perhaps the necessary road to truly practical AI Agents.