Marathon Runners vs Sprint Champions: Why AI "Endurance" Matters More Than "Intelligence"
> Paper: AutoLab: A Benchmark for Ultra Long-Horizon Closed-Loop Optimization > arXiv: 2606.05080 > Authors: Zhangchen Xu, Junda Chen, Yue Huang, et al. (20 authors) > Released: 2026-06-03
---
Introduction: Two Chess Players, Two Different Fates
Imagine two chess players. One is a prodigy who can calculate a winning sequence ten moves ahead at a glance. The other is of average talent, but has one special habit: after every move, they re-examine the entire board and ask, "Did this move actually improve my position?"
The prodigy wins the first game decisively. But what about the second game, the third, or the hundredth? When the game becomes long and the opponent learns the prodigy's patterns, the reflective player may well be the one laughing last.
This is not just a parable. It captures a striking finding about AI published on June 3, 2026 by a team of 20 researchers.
---
AutoLab: A Testing Ground for "Endurance"
Sprints vs. Marathons
In recent years, AI research has resembled a sprint championship. Researchers ask: can AI write perfect code in one shot? Solve a hard math problem in a single turn? Get the right answer on the first try?
These tests are fine—like the 100-meter dash, they measure "explosive power." But the real world is a marathon. A scientist does not complete a breakthrough experiment in one attempt; they hypothesize, experiment, fail, revise, and repeat. An engineer iterates on code, profiling bottlenecks, rewriting, and re-testing. Science and engineering are fundamentally about sustained iterative optimization.
36 Real Tasks, Four Demanding Domains
The AutoLab team built a benchmark of 36 real tasks across four domains, each requiring continuous closed-loop optimization under a strict time budget:
- Systems optimization: improve a poorly-performing software system or database by repeatedly modifying configurations and measuring performance.
- Puzzles and challenges: from classic algorithmic puzzles to logic games—the initial solution works but is far from optimal.
- Model development: train a better ML model from scratch or from a simple baseline, iterating on hyperparameters, architecture, and data preprocessing.
- CUDA kernel optimization: modify low-level GPU code for speed, requiring simultaneous understanding of algorithms, hardware architecture, and parallel computing.
- Gave up too early — tried a few times and stopped, as if saying "that's it then"; or
- Burned time blindly — used up the budget with negligible progress, going in circles.
- Local optima: Many models find a "decent" solution and keep polishing it, unable to escape the region to find a fundamentally better approach—like optimizing the horse cart instead of inventing the car.
- Credit assignment: Over 20 rounds of edits yielding a gain from 60% to 85%, which change mattered? Without answering this, AI cannot learn which modifications are effective and reuse that experience.
- Exploration vs. exploitation: With limited time, should the agent keep optimizing the current solution (exploit) or try something completely different (explore)? Excellent long-horizon strategies require dynamic balance between the two—a skill where current AI remains weak.
- Process reward models (PRM): reward each step of the optimization process, not just the final result.
- Metacognition: enable AI to monitor whether its own optimization process is working and adjust strategy accordingly.
- World models: let AI predict "what happens if I make this change?" and simulate internally instead of always executing.
- Xu, Z., Chen, J., Huang, Y., et al. (2026). *AutoLab: A Benchmark for Ultra Long-Horizon Closed-Loop Optimization*. arXiv:2606.05080.
- Sutton, R. S., & Barto, A. G. (2018). *Reinforcement Learning: An Introduction*. MIT Press.
- Lohn, A. J. (2023). *Kaizen: The Japanese Method for Transforming Habits*. Penguin.
- Hemingway, E. (1952). *The Old Man and the Sea*. Charles Scribner's Sons.
---
A Counterintuitive Finding: Persistence Beats Brilliance
The team evaluated 17 state-of-the-art models. The decisive factor for success was not initial capability but a "sustained iteration" character. Claude Opus 4.6 performed best—not because its first attempts were the most brilliant, but because it was the most "stubborn": after initial failures, it kept testing, modifying, and retrying until the time budget ran out.
Most other models either:
It's like two students: one brilliant but easily discouraged, the other less gifted but dogged. The first may win exams; the second almost always wins real-world complex problems.
Why Do AI Models Give Up So Easily?
Current language models are primarily trained to produce a correct answer in one shot. They rarely learn to approach correctness through repeated trial and error. A person raised only in "exam conditions"—one question, one attempt—might become an excellent exam machine but never an excellent scientist or engineer. They never learn to coexist with uncertainty, adjust direction after failure, or treat the process itself as learning. We taught AI "answers" but not the process of inquiry.
---
Time Awareness and Persistent Iteration
What Is "Time Awareness"?
AutoLab imposes strict wall-clock budgets, forcing decisions in real time. A capable agent knows how much time remains: with 3 hours left, it can try a complex strategy; with 5 minutes left, it should fall back to a simple one that guarantees a result. Many models lack this awareness entirely—attempting complex, time-consuming approaches as time expires, ending in total failure.
A Neuroscience Metaphor
Human brains solve complex problems with similar closed-loop mechanisms: when learning skills like cycling or guitar, the cerebellum continuously calibrates motor patterns through sensory feedback. Current LLMs, built on Transformers, resemble the cortex more than the cerebellum—strong at one-shot reasoning, weak at optimizing through feedback.
The Engineering Philosophy of Iteration
Toyota's Kaizen and agile software development both rest on small steps, continuous feedback, and iteration—acknowledging that no one can foresee all problems in advance. AutoLab reveals that AI needs a similar "cognitive humility": admitting it cannot get things right on the first try, and improving step by step through feedback.
---
Deep Challenges in Long-Horizon Optimization
---
Reflections: AI and "Craftsmanship"
Japan's *shokunin* concept—craftspeople who dedicate a lifetime to perfecting one skill—embodies treating the process as the goal itself. Hemingway's iceberg theory similarly points to the vast hidden revision beneath a clean finished surface. Perhaps future AI also needs a kind of craftsmanship: not "answer fastest" but "approach optimality through repeated refinement." AutoLab is trying to make AI excel in the underwater seven-eighths of the iceberg.
---
Outlook: When AI Learns to Persist
AutoLab points to several technical directions:
There is also a philosophical question: are our evaluation methods shaping the boundaries of AI capability? If we only test one-shot performance, AI evolves toward one-shot optimality. If we test sustained iteration, AI evolves toward endurance. AutoLab is a signal that the AI field needs more "marathon" tests, not just "sprint" tests.
---