Marathon Runners vs Sprint Champions: Why AI Endurance Matters More Than Intelligence

> Paper: AutoLab: A Benchmark for Ultra Long-Horizon Closed-Loop Optimization > arXiv: 2606.05080 > Authors: Zhangchen Xu, Junda Chen, Yue Huang, et al. (20 in total) > Released: 2026-06-03

---

Introduction: The Different Fates of Two Chess Players

Imagine two chess players.

One is exceptionally gifted, able to see a killing move ten steps ahead with a single glance. The other is of ordinary talent, but has a special habit: after every move, they re-examine the entire board and ask themselves, "Did this move really improve my position?"

In the first game, the genius wins. A fierce opening, a devastating middlegame, and the opponent collapses before they can react.

But what about the second and third games? What about the hundredth?

As the game grows long, as opponents learn their patterns, as initial brilliance is worn down by time — the player who reflects on every move may end up laughing last.

This is not a fable. It is a striking finding about AI, published on June 3, 2026, by 20 researchers.

---

AutoLab: A Testing Ground for "Endurance"

Sprints vs Marathons

Over the past few years, the AI field has resembled a grand sprint championship.

Researchers have focused on questions like: Can AI write perfect code in one shot? Can AI solve complex math problems in a single round of dialogue? Can AI give the right answer on the first try?

These tests are not wrong. They are like 100-meter sprints — pure, intense, and highly watchable. We do need to know how explosive AI's power is.

But the real world is not a sprint. The real world is a marathon.

A scientist never completes a breakthrough experiment "in one shot." They propose a hypothesis, run the experiment, find it's wrong, revise, run again, fail again, revise again... looping until some late night when the data finally lines up.

An engineer never writes optimal code "in one shot." They write a version, test it, find a bottleneck, rewrite, test again, optimize again... until an hour before the deadline, barely satisfied.

An entrepreneur never finds the right business model "in one shot." They try, fail, adjust, try again, fail again, adjust again... until after some pivot, they suddenly see the light.

This is the essence of science and engineering: continuous iterative optimization. Not reaching the top in one step, but being refined a hundred times over.

36 Real Tasks Across Four Demanding Domains

The AutoLab team understood this. Rather than designing yet another "one-shot Q&A" benchmark, they built an unprecedented test: 36 real-world tasks spanning four completely different domains, each requiring AI to perform sustained closed-loop optimization under strict time budgets.

What is "closed-loop optimization"?

Imagine you are debugging a complex machine. You make an adjustment, then measure the result. Not satisfied? Based on the feedback, you adjust again and measure again. The loop continues until the machine reaches its optimal state.

That's a closed loop. Not "fire and forget," but "adjusting the scope while shooting."

Each of AutoLab's four domains has its own brutality:

• System optimization: The AI optimizes a software system or database. The initial version runs, but performs poorly. The AI must repeatedly modify configurations, test performance, and modify again until it reaches the optimum — like a chef constantly adjusting a recipe, tasting, adding salt, tasting again — except the AI can only look at performance metrics.
• Puzzles and challenges: From classic algorithm puzzles to complex logic games. The initial solution is correct but far from optimal. Can the AI achieve genuine "refinement," like going from solving a Rubik's cube in 100 moves to 20?
• Model development: The AI trains a better machine learning model from scratch or from a simple baseline — tuning hyperparameters, modifying architectures, optimizing preprocessing. A graduate student might need months to raise a model's accuracy from 85% to 92%. Can AI do it in hours?
• CUDA kernel optimization: The deepest level of optimization. The AI modifies low-level GPU code to make computations faster, requiring simultaneous understanding of algorithms, hardware architecture, and parallel computing — and needs repeated trial and error to approach physical limits.

---

A Counterintuitive Finding: Persistence Beats Intelligence

Exam Results: 17 Models Put to the Test

The AutoLab team tested 17 state-of-the-art AI models — LLMs and agent systems of various sizes and architectures, from open-source to closed-source, from academia to industry.

The results were striking.

What determined success was not the models' initial capability, but whether they possessed the character of "sustained iteration."

Claude Opus 4.6 performed best. But not because it was the "smartest" — not because its first attempts were the most perfect. It was because it was the most "stubborn": even after initial failures, it would repeatedly test, modify, and retry until its time budget ran out.

And most other models?

They either gave up prematurely — trying a few times, failing, and stopping, as if to say, "Forget it, this will do." Or they burned blindly through their budgets — time ran out, but progress was minimal, as if spinning in place.

It's like the different fates of two students. One is smart but easily discouraged: when stuck on a hard problem, they try two approaches and give up. The other is less brilliant but tenacious: when the first approach fails, they try a second, then a third... until the twentieth, when they finally find the breakthrough.

In academic exams, the first student might win. But in real-world complex problems, the second student almost always wins.

Why Do AIs "Give Up Easily"?

This is a deep question. Why do advanced AI models get "discouraged" in long-horizon iteration?

The researchers believe it relates to how current AI is trained.

Today's language models are primarily trained to "generate the correct answer in one shot." They have read vast amounts of text and learned "what a good response looks like." But they are rarely trained to "approach the correct answer through repeated trial and error."

Imagine someone raised only in an "exam environment" — every question multiple choice, one chance per question. They might become an excellent exam machine, but never an excellent scientist or engineer.

Because they never learned how to live with uncertainty, never learned how to adjust course through failure, never learned to treat the process itself as an opportunity for learning.

This is the blind spot of current AI. We taught them "answers," but not the process of "seeking."

---

Time Awareness and Persistent Iteration

What Is "Time Awareness"?

AutoLab sets a strict "wall-clock budget" for each task. The AI cannot think forever; it must make decisions in real time.

This introduces a key variable: time management.

A good AI agent knows "how much time is left." With 3 hours remaining, it can attempt a complex strategy. With 5 minutes left, it should pick a simple strategy that guarantees at least a result.

It's like a student's exam strategy. With 30 minutes left on the last big problem, you can work through it carefully. With 5 minutes left, you should write a simplified version and grab partial credit.

The researchers found that many models completely lack this time awareness. As time runs out, they still attempt complex, time-consuming approaches, leading to total failure.

A Neuroscience Metaphor

Interestingly, the human brain uses a similar "closed-loop optimization" mechanism when solving complex problems.

Neuroscience research shows that when we learn new skills (like riding a bike or playing guitar), the brain constantly adjusts the activation patterns of motor neurons, correcting movements through sensory feedback (balance, hearing).

This process is not "computing the correct answer in one shot," but "constant trial, error, and adjustment."

The cerebellum is especially good at this closed-loop learning — a precision calibration system that turns clumsy movements into fluid skills through repeated practice.

Current AI, especially Transformer-based LLMs, resembles the cerebral cortex more than the cerebellum. They excel at "one-shot reasoning" but are poor at "sustained optimization through feedback."

The Philosophy of Iteration in Engineering

In real engineering practice, "iterative optimization" is everywhere.

Toyota Production System's "Kaizen" philosophy is built on small steps and continuous feedback. Workers are encouraged to propose small improvements daily, and over time, productivity rises dramatically.

Agile software development is also iterative: short "sprints" (usually two weeks), each ending with a working version, then feedback adjusts the next cycle's direction.

These methods work because they acknowledge human cognitive limits: we cannot foresee all problems at once, but we can keep learning and adjusting in practice.

AutoLab reveals a key issue: AI needs a similar "cognitive humility" — admitting it cannot get it right in one shot, and instead refining step by step through feedback like a human engineer.

---

Deep Challenges in Long-Horizon Optimization

The Local Optimum Trap

Optimization theory has a classic problem: local optima vs global optima.

Imagine walking in mountainous terrain, trying to find the highest point. But there's fog, and you can only see a few meters around you. You might climb a small hill and think you've reached the top, when a higher peak lies nearby — but you'd have to descend first to reach it.

Many AI models fail in AutoLab because they fall into "local optima." They find a "pretty good" solution and keep optimizing it, unable to step outside and search for an entirely different, possibly better one.

It's like someone optimizing carriage design to make it go faster — they will never invent the automobile. Sometimes real breakthroughs require abandoning the current approach and starting from zero.

The Credit Assignment Problem

Another key challenge in long-horizon iteration is "credit assignment": when a sequence of actions leads to a final result, how much did each action contribute?

Suppose the AI makes 20 rounds of modifications, improving performance from 60% to 85%. Which step deserves credit? The architecture change in step 3, the hyperparameter tweak in step 12, or the data cleaning in step 18?

If the AI cannot answer this correctly, it cannot learn "which modifications work and which don't."

It's like a chef who made 20 adjustments and finally created a perfect dish. If they don't know which adjustment was critical, they can't reuse that experience when cooking something new.

The Exploration-Exploitation Tradeoff

In reinforcement learning, there is a classic "exploration-exploitation tradeoff":

• Exploitation: continue doing what is known to work, maximizing short-term gains.
• Exploration: try new, uncertain approaches that may fail — but may also find something better.

In AutoLab's long-horizon optimization, this tradeoff becomes more complex. With limited time, the AI must decide: keep optimizing the current solution (exploit), or try something completely different (explore)?

It's like an investor deciding whether to put all their money in a stable fund (exploit) or take a portion and invest in a high-risk, high-reward startup (explore).

Excellent long-horizon optimization strategies require a dynamic balance between the two. AutoLab's results show that current AI is still very weak here.

---

Literary Reflections: AI and the "Craftsman Spirit"

The Japanese "Shokunin" Spirit

Japan has a cultural concept called "Shokunin" — craftsmen who dedicate their lives to one craft, pursuing perfection endlessly.

A sushi master might spend ten years learning to cook rice. Not because he is slow, but because he understands: ultimate sushi begins with ultimate rice.

The essence of this spirit is: treating the process as the goal itself. Not "I want to make sushi as fast as possible," but "I want to understand every possibility of rice."

AutoLab suggests an intriguing possibility: perhaps future AI also needs a kind of "craftsman spirit" — not pursuing "the fastest answer," but "approaching the optimum through repeated refinement."

Hemingway's Iceberg Theory

Hemingway said of writing: "The dignity of movement of an iceberg is due to only one-eighth of it being above water."

A good work looks clean and clear on the surface, but beneath lies an enormous process of preparation and revision. Hemingway reportedly revised the draft of *The Old Man and the Sea* more than 200 times.

If AI only sees the one-eighth above water — the final product — it may never understand the true creative process. It needs to see the seven-eighths below: the repetition, the revision, the discarding, the rewriting.

AutoLab is precisely an attempt to make AI excel in that "seven-eighths below water."

---

Future Outlook: When AI Learns to "Persist"

Technical Directions

AutoLab's results point to several important technical directions:

• Process Reward Models (PRM): reward every step of the optimization process, not just the final result — like a coach who watches not only the final score but also the effort and progress during training.
• Metacognitive ability: letting AI "think about its own thinking" — not just executing optimization, but monitoring whether the process is effective and whether strategy should change.
• World models: letting AI predict "if I make this modification, what will the result be?" so it can simulate mentally rather than executing every experiment.

Broader Implications

AutoLab's significance goes beyond a technical benchmark. It raises a philosophical question:

Is the way we evaluate AI shaping the boundaries of AI's capabilities?

If we only test "one-shot performance," AI will evolve toward "optimal one-shot performance." If we test "sustained iteration ability," AI will evolve toward "endurance."

It's like how an education system's evaluation criteria determine students' capability profiles. If the college entrance exam tests only multiple choice, students become good at quick judgments. If it tests research-based learning, students become good at deep inquiry.

AutoLab is a signal: the AI field needs more "marathon" tests, not just "sprint" tests.

---

References

• Xu, Z., Chen, J., Huang, Y., et al. (2026). *AutoLab: A Benchmark for Ultra Long-Horizon Closed-Loop Optimization*. arXiv:2606.05080.
• Sutton, R. S., & Barto, A. G. (2018). *Reinforcement Learning: An Introduction*. MIT Press.
• Lohn, A. J. (2023). *Kaizen: The Japanese Method for Transforming Habits*. Penguin.
• Hemingway, E. (1952). *The Old Man and the Sea*. Charles Scribner's Sons.