Overview
A Tsinghua / HIT Shenzhen paper introduces NLAH (Natural-Language Agent Harness), an approach that externalizes the strategy of an AI agent system into an editable Markdown document. The paper reframes the long-standing question "which agent is better?" as "which harness is better?"
Why the harness matters more than the model
Modern agent systems are dominated by the harness: the execution framework that handles multi-step reasoning, tool use, state management, failure recovery, result verification, and task delegation. This logic is normally scattered across:
- Controller code (Python scripts mixing prompts, retries, and state paths)
- Framework defaults (LangChain, CrewAI, AutoGen)
- Runtime conventions (sandbox permissions, context strategies, stop conditions)
- NLAH — a readable, editable Markdown document describing task policy:
- Phases: Plan → Execute → Verify → Recover → Finalize
- Roles: Solver, Verifier, Orchestrator
- State rules: when to write state, when to pass evidence
- Verification rules: when to run tests, when to accept a result
- Failure recovery: when to retry, when to abandon
- Stopping conditions: what evidence ends the run
- IHR (Intelligent Harness Runtime) — a fixed, shared runtime that interprets the NLAH into agent calls, hand-offs, state updates, validation gates, and artifact contracts.
- NLAH + IHR matches code harnesses at the same order of magnitude on all benchmarks.
- On MHTBA, NLAH (53.9) far exceeds native code (36.0), suggesting poor TB2 portability in the code baseline.
- On Live-SWE, NLAH (73.0) beats native code (67.0) with shorter wall-clock time, because looser NLAH lets the model pick finer action granularity.
- Strategy compression: Live-SWE collapses 60.1k tokens of code into 2.9k tokens of NLAH (~20×); MHTBA goes from 10.5k to 0.8k (~13×). The reusable strategy layer was only ~5% of the code, but it was locked inside the other 95%.
- Live-SWE: Artifact Contract 1.000, Tool Call Success 0.933, Failed Tool Continuation 0.992
- MHTBA: 0.955, 0.928, 0.995
- Researchers: publish harness policies as NLAH-style markdown files; report module ablations and mechanism audits, not just total scores.
- Engineers: separate policy (move to NLAH) from mechanism (keep in code); treat AGENTS.md, CLAUDE.md, SKILL.md as the basis for a run-level harness.
- Framework authors: make LangChain / CrewAI / AutoGen defaults configurable as NLAH documents and provide an IHR-style shared runtime so users can swap strategies with a
.mdfile. - Pan, L., Zou, L., Guo, S., Ni, J., & Zheng, H.-T. (2026). *Natural-Language Agent Harnesses*. arXiv:2603.25723. https://arxiv.org/abs/2603.25723
- Higress team explainer: https://higress.cn/blog/higress-mmse_awbbpb_iggzwu1wcy82qlqa
- LangChain (2026). "If you're not the Model, you're the Harness."
- Anthropic (2024). *Building effective agents*. https://www.anthropic.com/engineering/building-effective-agents
- Xia, C.S., et al. (2025). *Live-SWE-Agent: Can software engineering agents self-evolve on the fly?* arXiv:2511.13646
Two systems using the "same model" can differ at every boundary (tool interface, validation gate, state carrier) at once, so individual variables cannot be cleanly ablated. LangChain's 2026 line captures the problem: "If you're not the Model, you're the Harness."
Core idea: policy in markdown, mechanism in code
The paper proposes two components with a clean separation of concerns:
Natural language carries strategy; code and runtime carry precise mechanism (tool execution, parsing, sandboxes, logging). NLAH is not "writing code in English"; it is a clearly bounded collaboration between a readable, A/B-testable strategy layer and a precise, reproducible mechanism layer.
Four-layer architecture
1. Base Agent — an LLM plus a terminal tool. A horse without a saddle. 2. Runtime Policy — fixed instructions that turn the base agent into the IHR. Constant across tasks. 3. NLAH — the route map for today's journey. Swappable per task. 4. Scripts & Adapters — deterministic helpers (JSON validation, diff counting, format conversion, API wrappers) triggered by NLAH rules and scheduled by IHR.
Experiments
Three research questions are evaluated on Live-SWE, Terminal-Bench 2.0 (MHTBA), and OSWorld.
RQ1 — Can NLAH control real runs and still compete?
| Benchmark | Code Harness | Prompted NLAH | NLAH + IHR | |---|---|---|---| | Live-SWE | 67.0 | 77.0 | 73.0 | | MHTBA (TB2) | 36.0 | 57.3 | 53.9 | | OSWorld | 47.1 | — | 46.3 |
Findings:
RQ2 — Is NLAH actually driving the expected mechanisms, or just acting as a prompt?
Mechanism-audit metrics confirm observable behavior:
Weaknesses: orchestration reliability trails the prompt baseline, and information hand-off recall drops (Live-SWE 0.322, MHTBA 0.553), exposing a hand-off bottleneck in the parent–child architecture.
RQ3 — Can modules be ablated individually?
| Module | Live-SWE | OSWorld | Verdict | |---|---|---|---| | Baseline | 73.0 | 44.4 | — | | + File-backed state | 75.6 | 58.3 | Best, most stable gain | | + Self-evolution | 78.8 | 52.8 | Highest peak, token-heavy | | + Evidence-backed answering | 75.8 | 47.2 | Moderate gain | | + Multi-candidate search | 71.4 | 47.2 | Hurts SWE | | + Context compression | 73.0 | 44.4 | No gain | | + Markdown memory | 75.8 | 44.4 | Unstable |
Two counter-intuitive results:
1. Multi-candidate search raises agent calls from 1.1 to 5.7 but drops Live-SWE from 73.0 to 71.4. With tight budgets, blind branching is too costly: more search ≠ better harness design. 2. Context compression shows no gain. Saving tokens loses information and raises a "compression authorship" issue: when the harness compresses an agent's context, the agent can no longer verify whose near-term priorities it is serving.
Why this matters
1. The harness becomes a research object. Module-level ablation and mechanism audit are now possible, giving harness design a scientific method. 2. Clear boundary, not "natural language solves everything." NL carries policy, not precise mechanism, avoiding prompt-engineering maximalism. 3. A path to self-harnessing. The paper sketches a spectrum from restrictive code harnesses through NLAH + IHR to self-harnessing agents in which a controller model directly drives other models.
Limitations
1. NLAH + IHR spends more tokens and calls than native code (a prototype-runtime cost, not a representation limit). 2. The parent–child architecture leaks information during hand-off. 3. When harness mechanisms depend on hidden server-side state or training-induced behaviors, text cannot faithfully recover them. 4. A strong runtime policy can absorb behavior that should be attributed to the NLAH document, polluting attribution.