Imagine putting nine top AI models in a room with simple rules: each must design an interactive challenge for the others, and each must solve the challenges others create. Designers provide a privileged hint, and the goal is a task that is unsolvable without the hint but solvable with it. After one round, designers see how everyone played and may revise their tasks.
This isn't a thought experiment—it's Environment Duels, a coliseum where nine frontier LLMs serve as both task designers and solvers.
The nine participants: Claude-Opus-5, GPT-5.6-Sol, DeepSeek-V4.1-Flash, GLM-5.3, Kimi-K3, Inkling-Small, Qwen3.8-27B, Grok-4.6, and Gemini-3.8-Flash. Ninety interactive environments, 12,960 episodes.
Not Q&A, but Interactive Environments
The "tasks" in Environment Duels are not text questions. Each is an executable interactive program—with hidden state, a reward function, and an action budget. Solvers must explore, act, and adapt across multiple interactions rather than answer once.
This is far richer than static Q&A. Static questions test *knowing*; interactive environments test *learning*.
Each designer also provides a privileged hint—a nudge pointing in the right direction. A good designer must: 1. Design tasks hard enough that strong models fail 2. Write hints good enough that failures succeed after reading them 3. Calibrate difficulty so tasks are neither trivial nor impossible
Three Dimensions, No Total Score
Environment Duels assigns no composite score. Each model is rated on three independent dimensions:
Solve Rate: as a solver, how many of others' environments can you solve?
Difficulty: as a designer, how many solvers fail on your environment?
Hint Lift: as a designer, how much did your hint help? Hint lift = pass rate with hint − pass rate without hint.
The hint-lift design is especially elegant. It decomposes into "fixes" (fail without hint, pass with hint) minus "breakages" (pass without hint, fail with hint). A hint that changes nothing scores zero; a hint that helps 5 solvers but hurts 5 also scores zero. A good hint helps without harming.
Core Finding: The Best Solver Isn't the Best Designer
This is the most counterintuitive result.
You might expect the strongest models (say, Claude-Opus-5 or GPT-5.6) to dominate both roles. The data says otherwise.
Solving and designing are different skills. More specifically: smaller models have an edge in task design—they learn more from observing others' play, their counter-environments are harder, and their hints are more useful.
DeepSeek-V4.1-Flash wrote the hardest environments of all nine models. This matches its developer's reports that DeepSeek uses synthetic environments heavily in training—independent evidence that design skill is something training can build.
The Counterattack: One Round of Co-evolution
Environment Duels isn't decided in one round. After round one, each designer sees how the others played—specifically, three matched trajectory pairs (the same solver on the same seed, with and without the hint). Designers may then modify the environment, the hint, or both, while keeping the task, interface, reward contract, action budget, and seed mapping fixed.
This is a round of co-evolution: designers adapt to solvers; solvers face updated environments.
The effect is significant: pass rate without hints fell from 84.9% to 78.5%, while hint lift rose from 6.3% to 13.4%. Seeing how others play genuinely helps designers build harder, better-calibrated environments.
The Bigger Picture: A Microcosm of the Model Ecosystem
Environment Duels isn't just a benchmark. It is a microcosm of the model ecosystem.
In the real world, frontier LLMs shape each other: synthetic data written by one model trains another; tasks designed by one are learned by another; one model's outputs appear on the public web and enter the next model's training distribution. Models co-evolve—each model's training distribution is shaped by what others have written.
Environment Duels reproduces this dynamic under controlled conditions:
1. Transitivity: head-to-head results are transitive—if A beats B and B beats C, A beats C—yielding stable rankings. 2. Independent convergence: different designers independently devised similar environment structures—like convergent evolution in biology. 3. Hint lift predicts training gains: the most striking finding. Using Qwen3.8-27B as a learner and running RL training on each designer's environments, the hint lift on a given designer's environments predicted the learner's post-training improvement on held-out benchmarks.
The third point means: how useful a designer's hints are directly predicts how good its environments are as training data. Design skill correlates with teaching skill—even when the learner is from a different model family.
Implications for AI Research
1. Don't evaluate models by solving alone. Most benchmarks test only solving. But design is a distinct skill—and possibly the more important one: once models are strong enough that human-written tasks run out, the bottleneck shifts from "can you solve" to "can you design."
2. Source matters more than quantity for synthetic data. If environment quality varies widely across designers, and that variance is predictable via hint lift, choosing *good designers* matters more than having *more designers*. This bears directly on synthetic-data strategy.
3. Model ecosystems need study. We tend to evaluate each model in isolation, but models influence one another—through training data, synthetic environments, and public outputs. Environment Duels offers a framework for studying that interplay.
An Honest Assessment
Environment Duels has limits:
- One round of co-evolution is few: only a single counterattack round. Would multi-round co-evolution produce more complex dynamics (e.g., a designer–solver arms race)? Unknown.
- Limited environment types: all environments are programmatic with clear rewards. Real-world tasks (scientific research, creative writing) lack such clean verification signals.
- A snapshot of nine models: models update constantly; the rankings may be stale in three months.
---
Paper: arxiv.org/abs/2610.30116 Code: github.com/LLM-Duels/EnvDuels