Tev: Teaching AI to Run the Long Race — Training Long-Horizon Agents with Dense Progress Rewards
> When an AI agent needs to work independently for hours or even days to complete an economically valuable task, what capabilities does it need? A team from Adobe Research and UCLA offers their answer: a 'world model' that can precisely measure progress at every step.
A Simple Analogy: GPS Navigation vs. End-Point Rewards
Imagine driving from Beijing to Lhasa. If your GPS only says "You've arrived" at the destination and gives no feedback along the way, what happens? You don't know where you are, whether you've gone the wrong way, or how much longer it will take. All you can do is keep driving and hope.
This is the predicament facing current long-horizon agent training.
In reinforcement learning, the reward signal is the model's only guide. But long-horizon tasks have a fatal problem: the reward is too sparse. For a task requiring 200 steps, the model may receive no feedback for the first 199 steps, learning only at the final step whether it was "right" or "wrong." The paper calls this problem "Terminal Reward Starves Group-Relative RL."
Why "starves"? Because GRPO (Group Relative Policy Optimization) needs to compare the quality of different trajectories within a batch to assign rewards. If all trajectories only receive a 0 or 1 at the very end, the model cannot distinguish a trajectory that went 99 steps and fell one short from one that gave up after 10 steps. The signal is too weak.
Tev's Solution: First Build a Hidden "Perfect World"
Tev's core innovation is a "latent-world-first" task synthesis method.
The traditional approach: find a task first, then figure out how to design a reward for it. The problem is that rewards for long-horizon tasks are extremely hard to design.
Tev flips it around: first construct a "latent world" — a complete, correct, hidden task specification, including:
- A full description of the task domain
- Every step of the workflow
- What state the system should be in after each step
- 1,170 verified task environments across 7 domains
- Median agent rollout duration of 12.4 minutes — 6x the previous longest corpus (SWE-Smith, at 2.0 minutes)
- Every task comes with verifiable, dense, progressive rewards
- Pass@5 of 0.28, a 65% improvement over the Qwen3.8-27B baseline
- Matches GPT-6 Astra and Claude Opus 5
- TB 4.0-CM Pass@1 improved from 0.04 to 0.10, a 2.5x gain
- Reaches Claude Opus 4.8-level performance on TB 2.1
- Surpasses GLM 5.2 and Gemini 3.7 Flash
The agent-facing environment is then "rendered" from this latent world. It's like writing a detailed architectural blueprint first, then building the house from it — the blueprint is perfect because you wrote it yourself, and the house's complexity can be adjusted as needed.
The benefits are twofold: 1. Verifiable: because the latent world knows the correct state at every step, it can precisely determine where the agent currently is 2. Controllable complexity: tasks of different difficulty can be rendered from the same latent-world specification
Progress Potential: Turning "Where Am I?" into a Dense Reward
With the latent world in place, Tev introduces Progress-Potential Reward Shaping (PPRS).
The principle is intuitive: since the latent world knows the correct state at every step, it can compute the "distance" between the agent's current state and the correct state. That distance is the "progress potential" — the closer to the goal, the higher the potential. The change in potential at each step is that step's reward.
It's like a GPS telling you "800 km remaining" → "799 km remaining" every kilometer — feedback at every step, so the model knows it's moving forward.
A key mathematical detail: PPRS uses the potential-based reward shaping framework, which guarantees optimal policy invariance — adding this dense reward does not change the optimal policy the model ultimately learns; it only makes learning faster. This is an important theoretical guarantee, not an ad hoc engineering trick.
Data Scale: 6x Longer Than Prior Longest Tasks
Tev-Gym's key numbers:
This "6x" matters. Training data for long-horizon agents has long been a bottleneck: real-world long-horizon tasks are too expensive, slow, and hard to annotate. Through latent-world synthesis, Tev has for the first time pushed training data "length" to the 10-minute scale.
Experimental Results: An Open 27B Model Matches GPT-6 and Claude Opus 5
The trained Tev-27B model, evaluated on the computing and mathematical science domains of Agents' Last Exam (ALE-CM):
Even more notable is generalization: although Tev-27B was only trained on ALE-related data, it also performs strongly on Terminal-Bench (TB) 3.0 and 4.0 subsets:
The paper also mentions a Tev-320B scale, hinting the approach can continue to scale.
Why This Matters: A Prerequisite for RSI
The paper's abstract contains an easily overlooked but crucial statement: long-horizon agent capability is a "core prerequisite for automated discovery and recursive self-improvement (RSI)."
RSI — recursive self-improvement — is one of the most exciting and most concerning directions in AI safety research. An AI system that can improve itself needs to work independently for long stretches: analyzing its own code, designing experiments, running them, evaluating results, and modifying itself. Each step may take minutes to hours.
Tev addresses the weakest link in this chain: how to train models for long-duration, multi-step tasks requiring continuous judgment of progress.
Technical Highlights Recap
| Dimension | Traditional Approach | Tev's Approach | |-----------|---------------------|----------------| | Task source | Collected real tasks | Latent-world synthesis | | Reward design | Terminal reward | Progress potential (PPRS) | | Reward density | Sparse (final step only) | Dense (every step) | | Task verification | Manual or heuristic | Precise via latent world | | Task length | ~2 minutes | ~12 minutes |
An Honest Assessment
Tev is not without limitations:
1. Latent-world synthesis relies on domain knowledge: a complete task specification must be constructed first, which may not suit some domains (e.g., open-ended exploration) 2. Are 7 domains diverse enough: the paper does not discuss domain coverage boundaries in detail 3. 12 minutes is still far short of the true "hours to days" long horizon: the vision in the title remains aspirational; Tev's data only reaches the 10-minute scale 4. Code and data "coming soon": the project page says "Code soon, Data soon (under internal approval, will release in 7 days)" — verification will be needed upon release
But as a methodological validation of "how to train long-horizon agents," Tev delivers a solid result. The combination of latent-world-first synthesis + progress-potential reward shaping may become one of the standard paradigms for future long-horizon agent training.
---
Paper: Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards Project page: tev-agent.github.io Authors: Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh, Nanyun Peng, Kai-Wei Chang, Yong Jae Lee (Adobe Research / UCLA) Code/Data: Coming soon