> Paper: Code as Agent Harness: A Survey > arXiv: 2605.18747 > Authors: Xuying Ning, Katherine Tieu, Dongqi Fu, et al. (University of Illinois, Stanford, Meta, etc.) > Link: https://arxiv.org/abs/2605.18747
---
The Problem: Four Predicaments of Natural-Language Agents
Purely natural-language-driven AI agents are hitting an invisible wall.
The hallucination trap. Natural language has no compiler — errors don't get flagged. An agent can generate a "plausible-sounding" plan and only discover at step three that step one was already wrong — by which point the error has cascaded.
Fuzzy execution boundaries. "Help me organize my files" — what does "organized" mean? By date? By project? By size? The semantic gaps in natural language leave the agent and the user talking past each other.
No machine-checkable feedback. Whether execution results are good depends entirely on another LLM "getting a feel for it." No unit tests, no type checks, no deterministic verification — the whole system runs on vibes.
Catastrophic forgetting and state loss. Over multiple turns, the agent forgets decisions it made ten minutes ago. State is scattered across conversation history with no structured storage and no version management.
The survey's core thesis is blunt: in the LLM era, we need a new set of physical laws to constrain agents. And that law is code.
---
The Core Shift: From "Generating Code" to "Code as the Substrate"
In the past, the AI's job was to write code — once written, the task ended. Now, code is the underlying infrastructure through which agents perceive the world, reason, and act.
Code becomes the only deterministic medium connecting LLMs to the real world. It plays three roles simultaneously:
| Role | Function | |------|----------| | Vehicle for reasoning | Translates fuzzy intent into precise algorithmic steps | | Execution contract | Every statement has deterministic semantics, strictly checkable by interpreters/compilers | | Container for state | Variables, data structures, version control — state no longer scatters across conversation but accumulates in a code repository |
---
Three-Layer Architecture: Code as Agent Harness
The survey systematizes this paradigm into three interrelated layers.
Layer 1: Harness Interface
How code connects agents to the outside world:
- Connecting reasoning: the LLM outputs natural-language intent; code translates it into an executable plan
- Connecting action: function calls, API requests, file operations — all actions are issued as code
- Connecting environment modeling: code itself is a structured description of the environment (DOM trees, database schemas, system state)
- Planning: decomposing complex goals into verifiable chains of subtasks
- Memory: the code repository as long-term memory — commit history, branches, and diffs are all traceable memory traces
- Tool use: external tools invoked via typed code interfaces (APIs), with input/output constraints
- Feedback-driven control: execution results (success/failure/exceptions) feed directly back into the code layer, triggering retries, rollbacks, or replanning
- Optimization: code self-improvement based on execution history (e.g., automated refactoring, performance tuning)
- Multi-agent coordination: multiple agents operate on the same repository, resolving conflicts via PRs, branches, and merges
- Code review: one agent generates code, another reviews it (humans too)
- Verifiable shared state: a git commit hash serves as a state fingerprint — multi-agent shared deterministic snapshots
- Paper: https://arxiv.org/abs/2605.18747
- Authors: Xuying Ning, Katherine Tieu, Dongqi Fu, Tianxin Wei, Zihao Li, Yuanchen Bei, Jiaru Zou, Mengting Ai, Zhining Liu, Ting-Wei Li, Lingjie Chen, Yanjun Zhao, Ke Yang, Bingxuan Li, Cheng Qian, Gaotang Li, Xiao Lin, Zhichen Zeng, Ruizhong Qiu, Sirui Chen, Yifan Sun, Xiyuan Yang, Ruida Wang, Rui Pan, Chenyuan Yang, Dylan Zhang, Liri Fang, Zikun Cui, Yang Cao, Pan Chen, Dorothy Sun, Ren Chen, Mahesh Srinivasan, Nipun Mathur, Yinglong Xia, Hong Li, Hong Yan, Pan Lu, Lingming Zhang, Tong Zhang, Hanghang Tong, Jingrui He
Layer 2: Harness Mechanisms
Keeping the harness reliable and adaptive over long-running execution:
Layer 3: Multi-Agent Extension
From single agent to multi-agent, shared code artifacts become the coordination substrate:
---
Application Landscape
The survey maps seven domains where Code as Agent Harness is already landing:
1. Coding assistants: GitHub Copilot, Cursor, etc. — themselves prototypes of the harness 2. GUI/OS automation: controlling OS APIs via code, simulating keyboard/mouse, replacing brittle coordinate-based RPA 3. Embodied intelligence: robot control programs generated as code, pre-verifiable in simulators 4. Scientific discovery: experiment protocols described as code — reproducible and version-controlled 5. Personalized recommendation: user profiles and recommendation logic expressed as code rules — auditable and modifiable 6. DevOps: the CI/CD pipeline itself is a harness — code defines deployment, testing, rollback 7. Enterprise workflows: business processes orchestrated as code, not black-box "intelligent process engines"
---
Unsolved Challenges
The survey candidly lists six open problems in harness engineering:
1. Evaluation beyond final task success: were intermediate steps correct? Does partial completion have value? Existing benchmarks can't tell. 2. Verification under incomplete feedback: environmental feedback may be noisy, delayed, or even wrong — how does the harness stay robust? 3. Regression-free harness improvement: when an agent modifies its own code, how do you avoid "fix one bug, introduce three"? 4. Multi-agent shared-state consistency: concurrency control, conflict resolution, and eventual consistency in distributed repositories are far harder than for a single agent. 5. Human oversight of safety-critical actions: when an agent is about to drop a production database, transfer money, or ship code, how do you design "human-in-the-loop" interception? 6. Multimodal extension: current harnesses mostly target text/code environments — how do you incorporate images, audio, and physical-world perception?
---
Why It Matters
This is not another "agents that write code" paper. Its paradigm shift is this: code is no longer the agent's output target — it is the agent's runtime environment.
Natural language is intuitive, flexible, and suited to exploration. Code is precise, verifiable, and suited to execution.
Agents need both: natural language to understand intent, code to guarantee execution. Code as Agent Harness is exactly that bridging layer.
While the industry is still debating "are agents just hype," this survey has already drawn a clear technical roadmap — executable, verifiable, stateful AI agents must be built with code as their skeleton.
---