OpenThoughts-Agent: The 'Secret Recipe' for Training AI Agents
Paper: OpenThoughts-Agent: Data Recipes for Agentic Models Authors: Negin Raoof, Richard Zhuang, Marianna Nezhurina, et al. arXiv: 2606.24855 Published: 2026-06-23
---
The Kitchen Metaphor
Imagine you are a Michelin-starred chef creating a new dish. You could throw in every tasty ingredient by intuition and hope for the best — a likely disaster. Or you could run systematic experiments: vary one ingredient at a time, measure precisely, and converge on the optimal recipe.
That second approach — the scientific method — is the core philosophy of OpenThoughts-Agent. The project asks: how do we train a *broadly capable agent* — one that can do math, write code, search the web, and use tools — rather than a specialist in a single domain? Their answer: run 100+ controlled experiments to find the optimal data recipe, rather than piling up data on intuition.
---
Why Is Agent Training So Hard?
Unlike chatbots, agents can use tools (calculators, search engines, code interpreters), interact with environments (browsing, files, commands), perform multi-step reasoning, and make autonomous decisions. But this makes training much harder than standard LLM training:
- Interactive trajectories: training data is action → feedback → adjustment sequences, not static text.
- Multimodal: text, code, tool outputs, web content.
- Long-horizon: a task may involve dozens of interaction steps.
- Sparse feedback: correctness is often only known at the end.
- Error accumulation: one wrong step can derail everything downstream.
- Task diversity beats task count: with the same total data volume, mixed-task data outperforms the best single-task data by 3–5 points of average accuracy across 7 agent benchmarks.
- Don't over-filter: too-strict filtering wastes valuable data; too-loose filtering adds noise. There is a sweet spot that keeps "imperfect but useful" samples.
- Reasoning-trace quality is critical: the quality of chain-of-thought trajectories (clarity, logical coherence, self-correction) matters more than answer correctness — agents learn *how to think*, not just *what to do*.
- Strong scaling: more data consistently improves performance, with no saturation observed, and OpenThoughts-Agent data outperforms other open datasets at every training scale.
- OpenThoughts-Agent-32B: 44.8% average accuracy on 7 agent benchmarks
- Nemotron-Terminal-32B (previous strongest open agent model): 40.9%
- Improvement: +3.9 points — significant at equal model scale, especially given cross-domain generality.
- Benchmark artificiality: 7 human-designed benchmarks may not reflect real-world open-ended agent tasks.
- Cost: generating high-quality traces with frontier models is expensive.
- Hallucination: agents can still invent tools or fabricate results.
- Safety and alignment: more capable autonomous agents carry greater risk.
- Raoof, N., Zhuang, R., Nezhurina, M., et al. "OpenThoughts-Agent: Data Recipes for Agentic Models." arXiv:2606.24855, 2026.
- SWE-Smith, SERA, Nemotron-Terminal, Qwen3 Technical Report
Existing open datasets — SWE-Smith (software engineering), SERA (search-augmented reasoning), Nemotron-Terminal (terminal operations) — each target a single task type. Real users, however, pose problems that may require any combination of search, computation, coding, and reasoning. The core question OpenThoughts-Agent addresses: how to build a dataset that yields cross-domain agentic competence?
---
Methodology: 100+ Controlled Experiments
The main contribution is methodological. Every stage of the data pipeline was systematically ablated:
1. Task sourcing — math competitions (AIME, MATH), coding (Codeforces, LeetCode), science (GPQA, MMLU), synthetic tasks, real web-scraped tasks. 2. Quality filtering — what thresholds remove noise without discarding useful data? 3. Reasoning generation — how to produce high-quality multi-turn trajectories (think → act → observe → think again)? 4. Data mixing — what proportions of math/code/science data work best? 5. Training strategy — SFT, RL, curriculum choices.
Key Findings
---
Results: Qwen3-32B's Transformation
Fine-tuning Qwen3-32B with the best recipe yields:
Strengths across task families:
1. Math reasoning (MATH, AIME) — effective multi-step derivations with tool use. 2. Coding (Codeforces, LeetCode) — not just correct code, but self-correction after failed tests. 3. Science reasoning (GPQA) — PhD-level questions requiring deep reasoning and retrieval. 4. Tool use (web browsing, terminal) — learned *when* to search, compute, or reason directly.
Scaling experiments showed monotonic improvement from 10K → 50K → 100K samples, with OpenThoughts-Agent data beating other datasets at every scale — evidence that data quality (the recipe) matters more than raw quantity.
---
What Makes the Recipe Work
1. Multi-source task sourcing for diversity. 2. Quality-driven filtering — multi-metric screening without over-pruning. 3. Multi-turn reasoning generation — strong teacher models generate detailed, agent-style interaction trajectories, with best-of-N selection. 4. Balanced data mixing — proportions weighted by task difficulty and importance. 5. Iterative optimization — 100+ experiments tuning thresholds, mixing ratios, and generation parameters.
---
From Alchemy to Chemistry
The deepest takeaway is the *methodology*. AI training has long resembled alchemy — intuition-driven tuning with poorly understood outcomes. OpenThoughts-Agent represents a shift toward science: we now know which factors matter (task diversity, reasoning quality), which don't, where the sweet spots are, and what still scales.
The team also open-sourced everything: data, pipeline, experiment logs, and models — a strong example of open science in an increasingly closed industry.
---
Limitations and Future Directions
Future directions include multimodal agents, long-term memory, multi-agent collaboration, and lifelong learning.
---