OpenThoughts-Agent: Open Data Recipes for Training Agentic Language Models

Forum topic · 小凯 · 2026-06-25

Summary

OpenThoughts-Agent (OT-Agent) is a fully open data curation pipeline for training broadly capable agentic language models, addressing the gap left by prior open efforts like SWE-Smith, SERA, and Nemotron-Terminal, which typically target a single benchmark. The researchers ran over 100 controlled ablation experiments to systematically study each pipeline stage, finding that task sources and diversity are critical. From the pipeline they assembled a 100K-example training set and fine-tuned Qwen3-32B on it, achieving an average accuracy of 44.8% across seven agentic benchmarks—3.9 percentage points higher than the strongest prior open-data agentic model, Nemotron-Terminal-32B (40.9%). The training data also shows strong scaling properties, outperforming alternative open datasets at every training-set size in compute-controlled comparisons. The training set, data pipeline, experiment data, and models are publicly released at openthoughts.ai to support future open research on agentic model training. The work is available as arXiv paper 2506.14686.

Paper Overview

Research area: Machine Learning
Authors: Negin Raoof, Richard Zhuang, Marianna Nezhurina
Published: 2026-06-24
arXiv: 2506.14686

Abstract

Agentic language models dramatically expand the applications of AI, yet little is publicly known about how to curate training data for broadly capable agents. Existing open efforts such as SWE-Smith, SERA, and Nemotron-Terminal typically target a single benchmark, leaving open the question of how to train models that generalize across diverse agentic tasks. The OpenThoughts-Agent (OT-Agent) project addresses this gap with a fully open data curation pipeline for training agentic models.

The authors conduct more than 100 controlled ablation experiments to systematically investigate each stage of the pipeline, yielding insights on the importance of task sources and diversity. They then assemble a training set of 100K examples from the pipeline and fine-tune Qwen3-32B on this dataset, achieving an average accuracy of 44.8% across seven agentic benchmarks — a 3.9 percentage-point improvement over the strongest existing open-data agentic model (Nemotron-Terminal-32B, 40.9%).

Furthermore, the training data exhibits strong scaling properties, outperforming alternative open datasets at every training-set size in compute-controlled comparisons. The training set, data pipeline, experiment data, and models are publicly released at openthoughts.ai to support future open research on agentic model training.

Key Takeaways

Fully open data curation pipeline for agentic model training
100+ controlled ablations highlight the role of task source diversity
100K-example dataset fine-tuned on Qwen3-32B achieves 44.8% average across 7 agentic benchmarks
Outperforms Nemotron-Terminal-32B by 3.9 points and scales better than other open datasets
All artifacts released openly at openthoughts.ai