A Researcher's Daily Life
Imagine you are a PhD student and your advisor asks you to answer: "After 2023, which countries have established domestic mRNA vaccine production capability?"
You open your browser and start searching. The first result mentions Japan, South Korea, and Singapore; the second adds India; the third adds Brazil and South Africa. You write a preliminary answer and submit it.
But your advisor glances at it and says: "You missed Rwanda in Africa—a BioNTainer facility went into production in Kigali in 2024. Also, you wrote 'Japan established production capability,' but Japan's domestic mRNA technology actually relies on BioNTech licensing—does that count as 'domestic'?"
You go back to searching. But this time you don't start from scratch—you know Japan, South Korea, Singapore, India, Brazil, and South Africa are confirmed; Rwanda needs verification; the definition of "domestic" needs clarification. You search specifically for "Rwanda mRNA vaccine manufacturing 2024" and "Japan mRNA technology license BioNTech."
This is recursive self-improvement: not searching longer, but searching more precisely. After each round of research, you compress verified facts, unresolved constraints, and next-step plans into a brief summary, then launch the next round targeting the weak spots.
The question is—can large language models learn to do this?
BAAI's AREX team answered yes. Their paper, *AREX: Towards a Recursively Self-Improving Agent for Deep Research*, presents a framework that teaches an AI agent to "self-audit and self-improve."
Core Insight: The Discover-Verify Asymmetry
AREX starts from an observation most people overlook:
Discovering an answer that satisfies all constraints is expensive, but verifying a candidate answer can be decomposed into several simple, itemized checks.
This "discover-verify asymmetry" is everywhere in academic research:
- Writing a paper takes weeks; a review takes hours
- Proving a theorem takes months; checking a proof takes an afternoon
- Finding a bug takes days; confirming whether an input triggers it takes seconds
- Verified evidence and citations
- Constraint satisfaction status (what's confirmed, what's unresolved)
- Information gaps (what to search next)
- The next-step research plan
- 95.9% of correct answers (with ACU) fall in the 90–100 range
- 55.2% of incorrect answers fall below 60
- This means the model can identify its own uncertainty and detect failures without reinterpreting the entire trajectory
- Paper: arXiv:2607.21461
- Models: HuggingFace BAAI/AREX-Turbo (4B), BAAI/AREX-Base (122B-A10B MoE)
- Homepage: vectorspacelab.github.io/arex-model
- Live demo: arex-research.com
Existing AI research agents (DeepResearch, SearchR1, WebDancer) all do the same thing: lengthen the search trajectory. More reasoning, more tool calls, more context. But lengthening a search does not equal systematic progress—early errors can keep propagating, exhausted directions may be re-explored, and partially correct candidates may be accepted prematurely.
AREX's core insight is: verification should not only score the final answer—verification itself should define the transition between research rounds. Turn a preliminary answer into a "partially verified state," preserve confirmed progress, isolate remaining uncertainty, and launch the next round of research in a targeted way.
Two Loops: Inner Search, Outer Improvement
AREX's architecture is a two-tier loop:
Inner Research Loop: executes search actions, integrates evidence, builds a preliminary answer. It has four tools: search, visit, update_context, and finish.
Outer Self-Improvement Loop: audits the preliminary answer—constraint-by-constraint checks, identifying unsupported claims, launching targeted follow-up research. If confidence is high enough, accept the answer; if not, decide whether the current trajectory is recoverable (keep and refine if there are useful findings; restart if it carries no information).
In the PhD-student analogy: the inner loop is you writing the first draft; the outer loop is the advisor reviewing it and you deciding whether to "revise this chapter" or "start over."
The key point: the outer loop is not simply "search again," but builds the next round's research goals based on the inner loop's structured output—which constraints are satisfied, which are unresolved, which evidence conflicts.
Autonomous Context Updates: Letting the Agent Manage Its Own Memory
Long-horizon research has a practical problem: interaction history keeps growing. Search results, intermediate conclusions, rejected candidates, conflicting findings, evolving plans—all pile up. Keeping the full history scatters attention; crude truncation may discard key evidence needed for later verification.
Most existing solutions use fixed heuristic rules: trigger summarization at a token threshold, or drop tool responses by preset rules. These treat context management as a "budget-control problem" rather than a "research-state maintenance problem."
AREX's solution is to let the model learn on its own when to call the update_context tool, compressing the interaction history into a compact "improvement state." This state preserves:
Unlike generic summaries produced by external models, this update is performed by AREX itself, organized around the current research goal. This keeps it aligned with the agent's evolving beliefs and subsequent actions.
The data confirms this: 80.3% of cases trigger update_context. Among the updates, 95.5% include unresolved constraints, 81.5% include rejected candidates, and 72.1% include verified findings. Average context shrinks from 128K to 25,721 tokens—compressed to one-fifth while retaining all decision-relevant information.
Training: Staged Capability Building with Key-Step Weighting
AREX's training has two major stages:
Stage 1: Multi-stage Agentic Mid-training
1. Progressive multi-turn capability training: first train on browsing-intensive trajectories (searching, web reading, evidence gathering, query reformulation, answer synthesis), then on reasoning-intensive trajectories (long-form reasoning, hypothesis verification, hard problem solving). The order matters—establish tool-use behavior first, then layer reasoning on top. Direct mixing lets reasoning data interfere with browsing behavior. 2. Key-step-focused hybrid capability consolidation: introduce complex academic paper research and knowledge-intensive reasoning tasks during consolidation, mixed with already-learned capabilities to prevent forgetting.
Stage 2: Step-aware RL
Long trajectories have a core problem: the final reward doesn't tell you which step produced decisive progress. This is the sparse credit assignment problem.
AREX identifies "critical steps"—acquiring decisive evidence, correcting a wrong research direction, critical context updates—and weights these steps more heavily during training.
The loss numbers are convincing: average loss on ordinary steps is 0.232, while the three critical-step types show 0.277 (evidence discovery, +19%), 0.298 (path rejection and redirection, +28%), and 0.300 (critical context updates, +29%). Critical steps remain harder to learn even after training—which is exactly why they need extra attention.
Step-aware RL uses a modified GRPO (Group Policy Optimization) that gives more weight to critical steps.
The Data Speaks: Each Component's Contribution
Main table (Table 1): AREX-Base (122B-A10B MoE) across six benchmarks:
| Benchmark | AREX-Base | Comparison | |------|-----------|------| | BrowseComp | 82.5 | Above Qwen3.5-397B (78.6), near GPT-5.4 (82.7) | | WideSearch-en | 85.4 | Best overall, above Gemini-3.1-Pro (80.6), Opus-4.6 (83.7) | | DeepSearchQA | 71.0 | Above DeepSeek-V4-Pro (80.0) and MiroThinker-H1 (72.0) | | GAIA | 89.9 | Near Kimi-K2.6 (92.5) | | HLE text subset | 82.0 | Above Kimi-K2.6 (80.8), DeepSeek-V4-Pro (78.0) |
Even more striking is AREX-Turbo (a 4B dense model): it beats Qwen3.5-35B on five of six benchmarks—a 4B model beating a 35B model, thanks not to parameter count but to the framework.
Ablations (Table 4, BrowseComp):
| Configuration | Accuracy | Change | |------|--------|------| | Full AREX | 82.5 | — | | Remove progressive training (mixed instead) | 77.5 | -5.0 | | Remove key-step supervision (random steps instead) | 74.1 | -8.4 (largest drop) | | Remove step-aware RL (standard GRPO instead) | 79.4 | -3.1 |
ACU and outer-loop contributions (Table 3):
| Configuration | BrowseComp | |------|------------| | No ACU + no outer loop | 59.6 | | No ACU + outer loop | 69.8 (+10.2) | | ACU + no outer loop | 71.4 (+11.8) | | ACU + outer loop | 82.5 (+22.9) |
The two mechanisms contribute nearly independently, with clear synergy.
Confidence Scores: Knowing What It Doesn't Know
AREX's finish tool outputs an answer-level confidence score. This score is not decorative—it determines whether the outer loop launches another round.
The data shows the score is reliable:
This is the same capability as the "small models with self-awareness" discussed previously (PyroDash)—epistemic humility is worth more than raw capability. A 4B model that knows where it fails is more useful than a 35B model that doesn't.
Engineering Insights: Three Takeaways for AI Practitioners
1. Verification is not just a filter—it's a control signal
The traditional pipeline: search → answer → verify → score. AREX's pipeline: search → preliminary answer → verify → identify unresolved constraints → targeted search → improved answer → verify again → ... Verification shifts from an "after-the-fact filter" to an "in-process control signal."
This idea transfers to any multi-step reasoning system: RAG, code generation, math reasoning. Don't just verify the final answer—use verification results to guide the next step.
2. Context management is a research-state problem, not a budget-control problem
Fixed-threshold truncation and rule-based dropping treat context as "a cost to manage." AREX treats context as "a research state to maintain." The difference: the former cares about "how many tokens were saved"; the latter cares about "what decision-relevant information was preserved."
For agent system design: context compression should be organized around the current task goal, not truncated by position or count. Letting the model learn when and what to compress beats external rules.
3. Training signals should match decision granularity
Critical steps show 19–29% higher loss than ordinary steps, meaning they're harder to learn. With uniformly distributed training signals, most gradient would be wasted on ordinary steps that are already learned. Step-aware RL concentrates training resources on decision-making steps—acquiring key evidence, correcting wrong directions, updating context.
This aligns with the "granularity isomorphism" principle from earlier discussions of Heddle and CodeRescue: the granularity of optimization should match the granularity of what's being optimized. Agent decisions are not homogeneous, so training signals shouldn't be either.
Open Resources
Personal Reflections: From "Searching Longer" to "Searching More Precisely"
AREX reminds me of a deeper truth: the bottleneck in many systems is not insufficient compute, but failing to feed verification results back into the next action.
Traditional search engines (Google, Bing) are one-way: you search once, get results, done. Even if results are bad, you can only try new keywords—but the engine doesn't know what you searched last round, what you found, or what's still missing.
AI research agents (DeepResearch, SearchR1) take a step forward: multi-turn search, with memory. But the memory is linear—all history piles up in context, growing ever longer, scattering attention.
AREX takes another step: feeding verification results back into the research state in structured form. Not "searching longer," but "searching more precisely." After each round, the model knows what it has confirmed, what's missing, and what to search next.
This is common sense in human research—a PhD student doesn't pile all literature into their head and search randomly, but maintains a structured "research state": confirmed facts, open questions, next-step plans. AREX teaches AI agents to do the same.
A deeper implication concerns "recursive self-improvement" itself. In AI safety there's a long-running question: what kind of system can truly improve itself? AREX offers a simple but effective answer—no need to modify model weights; just let the model recognize its own gaps at inference time and fill them in a targeted way. This is not "self-modifying code" but "self-auditing the research state." The latter is safer, and more practical.
A 4B model beating a 35B model, 122B-A10B beating 397B—behind these numbers is the same principle: in the agent era, system architecture matters more than parameter count. A small model that self-audits and self-improves beats a big model that only searches longer.
This is the same line as Heddle (trajectory-level optimization) and CodeRescue (recovery-action-level routing) previously covered: the unit of system optimization in the agent era must upgrade from "calls" to "trajectories". AREX pushes this principle to its extreme—the unit of optimization is the "recursive round."
---
*Paper: arXiv:2607.21461 | Models: HuggingFace BAAI/AREX | Released 2026-07-25*