Harness-1: An Externalized-State Harness Lets a 20B Model Beat Closed-Source Giants on Search Agents

Forum topic · 小凯 · 2026-06-09

Summary

Harness-1 is an open-source search-agent framework built on the idea of state externalization: instead of forcing a policy model to both reason and manage search state inside its context window, the environment-side harness maintains a candidate pool, a curated set with importance labels, an evidence graph, verification records, and a token-budget marker. A 20B policy model then focuses purely on semantic decisions—what to search, keep, verify, and when to stop. Trained with SFT (899 filtered trajectories distilled from Kimi K2.5) followed by on-policy CISPO RL on only ~4.3K unique queries from a single SEC financial benchmark, Harness-1 achieves 73.0% average curated recall across 8 benchmarks, surpassing GPT-5.4, Sonnet-4.6, and Kimi-K2.5, with only Opus-4.6 (79.4%) higher. Notably, its zero-shot transfer gains (+17.0 pts over the Context-1 baseline) are 2.2x its in-domain gains, suggesting policies learn domain-general operations over an explicit search state. Ablations show all mechanisms contribute synergistically (fully removing them costs 12.2 points). Code and weights are open-sourced at github.com/pat-jj/harness-1; the paper is on arXiv (2606.02373).

Key points

Core idea — cognitive offloading: Harness-1 frees search agents from doing both "semantic decisions" and "state management." The environment-side harness (a structured state machine) manages the candidate pool, curated set, evidence graph, verification records, full-text store, search history, and token budget; the 20B policy model only decides what to search, what to keep/discard, what to verify, and when to stop.
Key mechanism — the Curated Set: The model maintains an explicit curated set of up to 30 documents with four importance levels (very high / high / fair / low), replacing implicit top-k truncation with explicit curation. Removing importance labels costs 4.1% recall.
Other harness components: an evidence graph (entity–document mapping for multi-hop bridging; removing it costs 2.6%), a verify tool with recorded verification outcomes (removing it costs 3.1%), Sentence-BM25 compression + MinHash dedup of candidates, and a token-budget marker ([Token usage: 14,203/32,768]) that makes the model budget-aware.
Training recipe:
Results:

| Model | Size | Avg. Recall | Type | |---|---|---|---| | Opus-4.6 | — | 79.4% | Closed frontier | | Harness-1 | 20B | 73.0% | Open, small | | GPT-5.4 | — | 70.9% | Closed frontier | | Sonnet-4.6 | — | 68.8% | Closed frontier | | Kimi-K2.5 | — | 64.7% | Closed frontier | | Context-1 | 20B | 61.6% | Open baseline | | Tongyi DR 30B | 30B | 60.3% | Open | | GPT-OSS-120B | 120B | 59.0% | Open | | Search-R1 32B | 32B | 28.9% | Open | | Qwen3 32B | 32B | 21.6% | Open |

Striking transfer result: RL training used only one benchmark (SEC financial reports), yet gains on 4 zero-shot transfer benchmarks (+17.0 pts over Context-1) are 2.2x the in-domain gains (+7.9 pts). The authors' interpretation: "the policy is learning operations over a domain-general search state" — explicit search state generalizes better than implicit dialogue history.
Ablations (BrowseComp+): importance labels −4.1%, evidence graph −2.6%, verify tool −3.1%, disabling everything −12.2% — mechanisms work synergistically, not as independent add-ons.
Data efficiency: only 899 SFT trajectories + ~4,352 unique RL queries, versus 8K (Search-R1) and 221K (Qwen3) for competitors. The trick: behavioral priors are moved into the interface, so RL optimizes only semantic decisions.

Acknowledged limitations

Scoped to evidence retrieval (needle-in-haystack / multi-hop), not open-ended report generation.
The evidence graph uses lightweight regex entity extraction — no entity disambiguation (e.g., aliases of the same person are not linked).
The LLM-based verifier can err, potentially giving false confidence on wrong evidence.
Sentence compression (top-4 sentences) can drop discourse-dependent context; full text is available via read_document but the token budget remains.
No built-in access controls for sensitive/copyrighted/biased content; filtering must happen at the application layer.

Takeaway

"The interface itself is part of the method." Rather than scaling model size, Harness-1 redesigns the agent's interaction surface so that everything that can be delegated to the environment is delegated. The same philosophy could apply to code agents (environment maintains AST/symbol tables), data-analysis agents (environment maintains schemas/statistics), or clinical agents (environment maintains patient timelines). Retrieval capability, the results suggest, is a function of interface design more than of parameter count.