OSU NLP Group, in collaboration with Amazon AGI SF Lab, has released QUEST — a fully open-source family of deep research agents spanning 2B to 35B parameters. Across 8 diverse benchmarks, QUEST performs on par with or better than closed-source frontier systems such as OpenAI DeepResearch and Kimi Researcher.
The headline number: it was trained on only 8,000 fully synthetic tasks.
Why This Matters: Pushing Against the Closed-Source Wall
Deep research agents are among the hottest areas of 2025–2026. Systems like OpenAI DeepResearch, Kimi Researcher, Manus Wide Research, and Anthropic's multi-agent research evolve search from "returning keyword-matched pages" to "autonomous investigation with fully cited reports."
But they are all closed-source: no weights, no data, no training recipes.
Open-source efforts (Tongyi-DR, OpenResearcher, SFR-DeepResearch, OpenSeeker) each cover only specific scenarios — some excel at fact retrieval, others at report generation — with no unified, cross-task open solution. QUEST aims to fill exactly that gap: a general-purpose, fully open, strong-across-task-types deep research agent.
Core Innovation 1: The Unified Rubric Tree
Deep research evaluation is naturally split:
- Objective tasks: externally verifiable answers (BrowseComp, GAIA)
- Open-ended tasks: multi-dimensional subjective judgment (DeepResearch Bench — coverage, organization, clarity, insight)
- Root: overall score
- Internal nodes: high-level constraints (e.g., "technical comparative analysis"), recursively decomposed
- Leaf nodes: directly verifiable fine-grained criteria (e.g., "cited Apple's foldable-screen patent"), each scored binarily
- Models: 2B/4B/9B/30B/35B, each with MT, SFT, MT+SFT, and RL checkpoints
- Data: RL data and both objective/open SFT data released
- Code: inference pipeline, evaluation scripts, training scripts (SFT on LlamaFactory, RL on VERL with Megatron, fully-async), and the data generation pipeline
- Config templates: API, endpoints, and eval LLM configurations
- Xie et al. (2026). QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks. arXiv:2605.24218.
- Project: https://osu-nlp-group.github.io/QUEST
- Code: https://github.com/OSU-NLP-Group/QUEST
- Models & Data: https://huggingface.co/collections/osunlp/quest
Existing open agents train on "complex question + single verifiable answer" data, which generalizes poorly and provides only sparse binary RL rewards.
QUEST's Rubric Tree is a hierarchical decomposition of "what a good answer means":
Example objective task: "Which of the 2024 US listeria outbreaks killed more people?" decomposes into identifying each outbreak (Boar's Head deli meat / 10 deaths; Rizo-Lopez cheese / 2 deaths) and comparing — every node independently verifiable.
Example open task: "Evaluate Apple entering the foldable phone market" — the root has four fixed shared criteria (Instruction Following, Comprehensiveness, Readability, Insight) with adaptive sub-criteria.
Benefits: one framework covers both task types; partial root-node scores give richer RL signals than binary correctness; GPT-5 automatically translates rubric trees into Python verification code, enabling scalable automation.
Core Innovation 2: Fully Synthetic Data Pipeline — QUEST-8K
QUEST's training data is entirely synthetic, with zero human annotation.
Objective task synthesis: trending keywords from Google Trends seed topics; Claude Sonnet 4.5 autonomously browses the web, collects information, and extracts verifiable constraints; rubric trees are built, iteratively refined, and self-validated; GPT-5 then generates corresponding Python evaluation scripts.
Open-ended task synthesis: rubric tree roots use the four fixed shared criteria; sub-criteria are generated adaptively with GPT-5-assigned weights (averaged over three generations for stability); Claude Sonnet 4.5 produces reference reports. Scoring uses pairwise normalization: the judge scores candidate and reference reports 0–10 each, with final score = J(candidate) / (J(candidate) + J(reference)). Scores above 0.5 mean the candidate beats the reference — avoiding score-scale drift across tasks of differing difficulty.
Core Innovation 3: Three-Stage Training
1. Mid-training (MT): adapts the base model (Qwen3-30B-A3B / Qwen3-35B-A3B) to the format and rhythm of long-horizon search interactions (search → read → reason → search again). 2. SFT: teaches the agent to imitate high-quality tool-use trajectories (objective tasks) and reference reports (open tasks). Notably, SFT-only small models are surprisingly strong — QUEST-2B-SFT reaches 30.3 on HLE and 72.8 on GAIA — showing much deep research capability can be injected via high-quality imitation alone. 3. RL: optimizes the policy with rubric-based partial scores at the root node, letting the model learn trade-offs among multi-dimensional constraints and surpass the imitation ceiling. Ablations show monotonic gains from +SFT → +MT → +RL.
Core Innovation 4: Context Management as a Cognitive State Machine
Long-horizon agents suffer from information forgetting. QUEST's Context Management maintains a structured Context State (JSON) with three buckets:
1. Trusted: facts verified via source URLs; reusable without re-verification. 2. Untrusted: claims contradicting other sources, with reasons for distrust; demoted unless new evidence demands re-checking. 3. Uncertain: partially supported claims, each annotated with URLs to follow up or queries to re-run.
When the context window exceeds a threshold, a Context Condenser (GPT-5-mini) updates the Context State from the full history; the agent then continues in a fresh window carrying only the structured state. Trusted entries avoid redundant API calls; Uncertain entries directly guide next actions; Untrusted entries prevent verification loops. On BrowseComp / BrowseComp-Plus, QUEST uses a "discard-all" strategy — relying entirely on the Context State for cognitive coherence.
Benchmark Results
| Benchmark | Type | QUEST-35B | vs. Closed-Source | |---|---|---|---| | BrowseComp | Fact retrieval | 64.6% | Close to OpenAI DeepResearch | | Mind2Web 2 | Citation verification | 30.7% | Standout | | DeepResearch Bench | Report synthesis | 48.2% | Matches or exceeds closed-source |
Controlled 30B-scale comparisons: Tongyi-DR is strongest on fact-retrieval benchmarks; OpenResearcher leads BrowseComp-Plus; QUEST-30B performs best on Mind2Web 2 and DeepResearch Bench — its cross-benchmark balance comes from the training recipe, not scale. Scaling from 2B→4B→9B→35B yields monotonically increasing performance on all benchmarks.
Why Only 8,000 Tasks Suffice
1. High-density supervision: each rubric tree contains 10–30 constraint nodes, so 8,000 tasks yield hundreds of thousands of fine-grained signals. 2. Task complexity: each trajectory may involve 10–50 steps of search, browsing, reasoning, and synthesis, amplifying effective data volume. 3. Quality over quantity: strict self-verification and iterative refinement filter for high-quality samples.
This aligns with a broader trend: synthetic data may follow different scaling laws, with structural design and validation letting small amounts of data produce outsized training effects.
Openness
Limitations
1. Cached search databases and mid-training data are still under legal review, affecting full reproducibility (especially RL requires pre-built search/scholar/visit databases). 2. Small models (2B–4B) still struggle with open-ended report synthesis. 3. Open-task evaluation depends on Claude/GPT-5 as judge, inheriting their biases. 4. The Context Condenser (GPT-5-mini) is a single point of failure for long-horizon coherence.
Bottom Line
QUEST unifies objective and open-ended task training under one rubric tree framework, transforms base models into deep research agents via three-stage training, solves long-horizon forgetting with structured context management, and matches closed-source frontiers on 8 benchmarks with just 8,000 synthetic tasks — all fully open-sourced.
It demonstrates that the deep research training recipe can matter more than model scale.
References: