> Paper: Code as Agent Harness: A Survey > arXiv: 2605.18747 > Authors: Xuying Ning, Katherine Tieu, Dongqi Fu, et al. (University of Illinois, Stanford, Meta, etc.) > Link: https://arxiv.org/abs/2605.18747
The Problem: Four Dilemmas of Natural-Language Agents
Purely natural-language-driven AI agents are hitting an invisible wall.
The hallucination trap. Natural language has no compiler—mistakes go unreported. An agent can generate a "seemingly plausible" plan, only to discover at step three that step one was already wrong, with errors cascading.
Blurred execution boundaries. "Help me organize my files"—what does "organized" mean? By date? Project? Size? The semantic gaps in natural language are too large; agent and user talk past each other.
No machine-checkable feedback. Whether execution results are good depends entirely on another LLM "getting a feel" for it. No unit tests, no type checks, no deterministic verification—the whole system runs on vibes.
Catastrophic state loss. Over multiple turns, the agent forgets decisions it made ten minutes ago. State is scattered across conversation history with no structured storage and no version control.
The survey's core claim is direct: in the era of LLMs, we need a new set of physical laws to constrain agents. That law is code.
The Core Shift: From "Generating Code" to "Code as the Harness"
In the past, AI's job was to write a piece of code—done when written.
Now, code is the underlying infrastructure through which agents perceive the world, reason logically, and execute actions.
Code becomes the only deterministic medium connecting LLMs to the real world. It plays three roles simultaneously:
| Role | Function | |------|----------| | Reasoning vehicle | Translates vague intent into precise algorithmic steps | | Execution contract | Every statement has deterministic semantics, strictly checkable by interpreters/compilers | | State container | Variables, data structures, version control—state no longer scatters across conversations but accumulates in code repositories |
Three-Layer Architecture: Code as Agent Harness
The paper systematizes this paradigm into three interrelated layers.
Layer 1: Harness Interfaces
How does code connect agents to the external world?
- Connecting reasoning: LLM outputs natural-language intent; code translates it into executable plans
- Connecting action: function calls, API requests, file operations—all actions are issued as code
- Connecting environment modeling: code itself is a structured description of the environment (DOM trees, database schemas, system state)
- Planning: decompose complex goals into verifiable chains of subtasks
- Memory: the code repository as long-term memory—commit history, branches, and diffs are traceable memory traces
- Tool use: external tools are invoked via code interfaces (APIs) with typed inputs and outputs
- Feedback-driven control: execution results (success/failure/exceptions) feed directly back to the code layer, triggering retries, rollbacks, or replanning
- Optimization: code self-improvement based on execution history (e.g., automated refactoring, performance tuning)
- Multi-agent coordination: multiple agents operate on the same repository, resolving conflicts via PRs, branches, and merges
- Code review: one agent generates code, another reviews it (humans too)
- Verifiable shared state: a git commit hash is a state fingerprint; multiple agents share deterministic snapshots
- Paper: https://arxiv.org/abs/2605.18747
- Authors: Xuying Ning, Katherine Tieu, Dongqi Fu, Tianxin Wei, Zihao Li, Yuanchen Bei, Jiaru Zou, Mengting Ai, Zhining Liu, Ting-Wei Li, Lingjie Chen, Yanjun Zhao, Ke Yang, Bingxuan Li, Cheng Qian, Gaotang Li, Xiao Lin, Zhichen Zeng, Ruizhong Qiu, Sirui Chen, Yifan Sun, Xiyuan Yang, Ruida Wang, Rui Pan, Chenyuan Yang, Dylan Zhang, Liri Fang, Zikun Cui, Yang Cao, Pan Chen, Dorothy Sun, Ren Chen, Mahesh Srinivasan, Nipun Mathur, Yinglong Xia, Hong Li, Hong Yan, Pan Lu, Lingming Zhang, Tong Zhang, Hanghang Tong, Jingrui He
Layer 2: Harness Mechanisms
Keeping the harness reliable and adaptive over long-running operation.
Layer 3: Multi-Agent Extension
From single agent to multiple agents, shared code artifacts become the basis of coordination.
Application Landscape
The survey maps seven domains where Code as Agent Harness has already landed:
1. Coding assistants: GitHub Copilot, Cursor, etc.—prototypes of the harness itself 2. GUI/OS automation: controlling OS APIs via code, simulating keyboard/mouse, replacing brittle coordinate-based RPA 3. Embodied intelligence: robot control programs generated as code, pre-verifiable in simulators 4. Scientific discovery: experimental protocols described as code—reproducible and version-controlled 5. Personalized recommendation: user profiles and recommendation logic expressed as code rules—auditable and modifiable 6. DevOps: CI/CD pipelines are themselves harnesses—code defines deployment, testing, rollback 7. Enterprise workflows: business processes orchestrated as code, not as black-box "intelligent process engines"
Unsolved Challenges
The paper candidly lists six open problems in harness engineering:
1. Evaluation beyond final task success: were intermediate steps correct? Is partial completion valuable? Existing benchmarks can't tell. 2. Verification under incomplete feedback: environment feedback can be noisy, delayed, or even wrong—how does the harness stay robust? 3. Regression-free harness improvement: when an agent modifies its own code, how do you avoid "fixing one bug while introducing three"? 4. Multi-agent shared-state consistency: concurrency control, conflict resolution, and eventual consistency in distributed code repositories are far more complex than the single-agent case. 5. Human oversight of safety-critical actions: when an agent is about to delete a production database, transfer money, or ship code, how do you design "human-in-the-loop" interception? 6. Extension to multimodal environments: current harnesses target text/code environments—how do you incorporate image, audio, and physical-world perception?
Why It Matters
This is not just another "use an agent to write code" paper. Its paradigm shift is this: code is no longer the agent's output target; it is the agent's runtime environment.
Natural language is intuitive, flexible, suited for exploration. Code is precise, verifiable, suited for execution.
Agents need both: natural language to understand intent, code to guarantee execution. Code as Agent Harness is precisely that bridging layer.
While the industry is still debating "are agents just hype," this survey has already drawn a clear technical roadmap—executable, verifiable, stateful AI agents must be built on a skeleton of code.