Code as Agent Harness: When Code Becomes the Nervous System of AI Agents
Paper: Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems Authors: Xuying Ning et al. | UIUC × Meta × Stanford Link: https://arxiv.org/abs/2605.18747
---
1. An Overlooked Perspective
When we discuss AI agents, attention habitually goes to the model itself—parameter count, reasoning ability, context length. But this paper raises a long-neglected question: if the LLM is the brain, what is its nervous system?
Traditionally, the answer has been natural language. The model generates text, an external system parses it, and then decides which tools to call or which files to modify. Text is the bridge, but it has a fundamental flaw—it cannot be executed.
Imagine a chef (the LLM) working in a kitchen. The traditional approach is for the chef to describe every step in natural language: "first chop the onion, then heat the pan, then pour the oil..." An assistant in the kitchen listens and executes. The problems: what if the description is vague? What if the assistant misunderstands? If a step fails, can the chef know immediately?
The paper's core claim is simple: let the chef write code directly. Not a recipe for humans, but a script for machines to execute. Chopping an onion becomes a function chop(onion), heating the pan becomes heat(pan, 180)—every step executable, verifiable, and rollback-able.
This is the intuition behind "Code as Agent Harness":
> Code is not the agent's output; it is the agent's nervous system—the medium connecting perception, reasoning, and action.
---
2. Why Code? Three Irreplaceable Properties
The paper presents a concise argument: natural language has three blind spots, and code fills them.
| Property | Natural Language's Problem | Code's Solution | |------|-------------|-----------| | Executability | Describing "compute the average" without specifying edge cases | Written as a function, inputs and outputs strictly defined at runtime | | Inspectability | Reasoning hidden in a black box; errors can only be guessed | Every step generates intermediate variables that the harness can read, verify, and score | | Statefulness | Prior conversation content may be lost or distorted | The program itself is the state carrier: variable values, execution traces, and error logs are all persistent |
These are not aesthetic judgments like "code is more precise." They are engineering constraints—when you want a system to run continuously for hours, span hundreds of steps, and self-repair after failures, natural language simply cannot hold up.
The paper distinguishes two easily confused concepts:
- System-provided harness: tools, APIs, sandboxes, and verifiers predefined by the system (e.g., a tool layer like OpenClaw's)
- Agent-initiated code artifacts: code objects the agent creates, modifies, and executes during operation (e.g., temporary scripts, regression tests, reusable skills, intermediate program state)
- Linear decomposition: break tasks into sequentially executed subfunctions, like writing functions
- Structural grounding: use code structure (classes, modules, dependency graphs) to organize planning
- Search-based planning: search possible execution trajectories in code space (analogous to AlphaGo searching the board)
- Orchestration-based planning: use workflow engines to coordinate execution order across code units
- Working memory: files currently being edited, open variables
- Semantic memory: codebase architecture knowledge, API documentation
- Episodic memory: past successful solutions, failed attempts
- Long-term memory: reusable skill libraries, regression test suites
- Functional tools: calling predefined APIs (like library functions)
- Environment-interaction tools: directly manipulating external environments (browsers, file systems, databases)
- Verification-driven tools: validating code via tests, type checking, static analysis
- Workflow-orchestration tools: coordinating execution order and dependencies across tools
- Plans as contracts: an agent's generated plan is itself a verifiable commitment
- Sandboxed execution: run agent code in isolated environments to protect production
- Deterministic sensors: measure state with objective metrics like test pass rates and compile error counts
- Harness-level optimization: optimize the whole agent system's execution strategy, not just model parameters
- Manager Agent: generates task-decomposition code (workflow definitions)
- Coder Agent: implements features
- Reviewer Agent: reads code and generates review comments (also code comments or tests)
- Tester Agent: runs tests and generates reports
- Centralized: a Manager Agent coordinates multiple Worker Agents
- Distributed: agents communicate directly, like microservices
- Streaming: one agent's output feeds directly into the next, forming a pipeline
The former is "the kitchen's infrastructure"; the latter is "notes and self-invented techniques the chef jots down while cooking." Existing research mostly focuses on the former; this paper turns the spotlight to the latter—agent-initiated code interactions.
---
3. Three-Layer Architecture: How Code Becomes the Agent's Nervous System
The paper organizes "Code as Agent Harness" into three interconnected layers. Each layer addresses a class of problems that arise as agents evolve from "single-shot generation" to "continuous operation."
Layer 1: Harness Interface
This is the question of "how code enters the agent loop," split into three roles:
1. Code for Reasoning
The problem with Chain-of-Thought isn't the ideas—it's that they cannot be verified. The model says "I computed the average," but did it really? Correctly?
The core insight of Program-of-Thoughts (PoT) and follow-up work (Chain of Code, CodePRM, RLEF, etc.): have the model generate executable programs and outsource reasoning to an interpreter. The model proposes strategy ("use dynamic programming"); the interpreter handles execution details ("array out of bounds? Return an error, let the model fix it").
Going further, systems like Lean4Agent bring formal proof into the agent loop—not "I think this strategy is right," but "the prover confirms this strategy is logically airtight."
2. Code for Acting
Agents can't just reason internally—they must act on the world: click buttons, move robotic arms, submit GitHub PRs. The traditional approach maps model outputs to a predefined action space, but that's rigid.
The direction the paper surveys: have agents generate programmatic policies. Not "call tool A," but "write a function that decides which element to click based on the current DOM state." These policies are executable, inspectable, and iterable.
Representative work includes Voyager's skill library (a Minecraft agent writing its own JS code to control its character), programmatic UI manipulation in GUI agents, and code-level action in systems like Claude Code and Codex.
3. Code for Environment Modeling
Agents need to understand their environment. Instead of having the model directly "read" the environment (screenshots, documents), the paper points to a more efficient approach: represent environment state as code.
A code repository is itself a structured world representation. Execution traces record the system state after each agent action. Test cases define the environment's "laws of physics"—if you change this line of code, which tests will fail?
Most elegant is Verifiable Environment Construction: agents can write code to build their own test environments—e.g., "I'll write a simulator to verify my policy is stable across different parameters."
Layer 2: Harness Mechanisms
Interfaces solve "how to connect"; mechanisms solve "how to keep running." Long-horizon tasks require:
1. Planning
Decompose large tasks into executable code blocks. Four planning paradigms:
2. Memory
Code is naturally a memory carrier:
The paper highlights a key problem: Context Compaction. When the agent runs long and the context window fills up, the answer is to "offload" state into code structures—turn conversation history into comments, intermediate results into variables, decision processes into Git commits.
3. Tool Use
Four categories of tool use:
4. Control & Optimization
The key to reliability is a "Plan-Execute-Verify" loop:
Most interesting is the Evolution Agent—an agent dedicated to improving the harness itself. It observes the main agent's execution traces, proposes harness improvements ("add a memory cache? switch planning strategies?"), then verifies improvements in a sandbox.
Layer 3: Scaling the Harness (Multi-Agent)
When multiple agents collaborate, the question becomes: how do they share the same nervous system?
The paper's answer: code itself is the shared medium.
Imagine a software team: a PM writes requirements (natural language), an architect draws designs (semi-structured), programmers write code (executable), testers write test cases (verifiable). Traditionally these are different media requiring constant translation and syncing. Under "Code as Agent Harness," all roles work in the same code repository:
They share the same "world representation"—Git repo state, CI/CD results, coverage reports. When one agent modifies code, others immediately see it via git diff.
The paper also discusses multi-agent topologies:
---
4. From Theory to Practice: Five Application Domains
1. Coding Assistants
Claude Code, GitHub Copilot Workspace, SWE-agent—these systems treat code as the harness. Agents don't just generate snippets; they plan, run tests, and submit PRs within real codebases.
2. GUI/OS Automation
Systems like Computer Use agents treat interface operations as code-level programs. Not "click at screen coordinates (x, y)," but "generate a program that decides the action sequence based on the current DOM state."
3. Embodied Intelligence
Robots and physical-world agents use generated programs as policies. Voyager's Minecraft JS skill library is a classic example—the agent doesn't just "remember how to mine"; it keeps an executable mine.js script.
4. Scientific Discovery
Agents build hypothesis-testing pipelines in code: generate simulation code, run experiments, analyze data, adjust hypotheses based on results. The scientific method becomes an executable program.
5. Personalization & Recommendation
Agents generate personalized code artifacts for users—custom scripts, config files, automated workflows. These are both solutions and persistent representations of user preferences.
---
5. Open Challenges: Seven Hard Problems
1. Harness-Level Evaluation
Current agent evaluation focuses on "did it get the final answer right." Real engineering questions: how many steps did it take? What mistakes occurred along the way? Can it recover automatically after failure? New metrics are needed to measure agent robustness, not just correctness.
2. Semantic Verification Beyond Executable Feedback
Code that runs isn't necessarily correct. A function passing tests may deviate semantically from the user's intent. How do you verify "this implementation actually satisfies the user's intent"?
3. Self-Evolving Harnesses without Regression
If agents can modify their own harness (Evolution Agents), how do you ensure improvements don't break existing functionality? A regression-testing problem, but happening at runtime.
4. Transactional Shared Program State
Multi-agent modifications to shared codebases should be atomic and rollback-able. The paper proposes transactional semantics—each agent step either fully applies or fully reverts.
5. Human-in-the-Loop Safety
When agents can execute code, safety is no longer "prevent the agent from doing bad things" but "how do humans always understand and intervene in agent decisions?"
6. Multimodal Code-Harness Systems
Code isn't only text—it can be visual programs (node graphs), tactile feedback, encoded voice commands. How do these non-text modalities fit the "Code as Harness" framework?
7. Toward a Science of Harness Engineering
Finally, the paper calls for a systematic discipline of "Harness Engineering"—not ad-hoc tool assembly, but a field with design principles, evaluation methods, and best practices, like software engineering or systems architecture.
---
6. Closing Thoughts
What's most striking about this paper isn't its technical depth (it is, after all, a survey) but its shift in perspective.
For years we've treated code as the LLM's "output target"—like a painter producing a painting. This paper reminds us: code can be the canvas itself. The agent paints on the canvas while observing its state, modifying its structure, and making its elements interact.
Claude Code works well not because its generated code is prettier, but because it turns the entire development environment into the agent's nervous system. The code repository is memory, the terminal is the action interface, tests are verification sensors, LSP is the environment model.
Behind this lies a deeper shift: AI is moving from "generating content" toward "becoming an operating system." Not handing you text to read, but giving you an executable environment to use.
Reference Paper:
Ning, X., Tieu, K., Fu, D., et al. (2026). Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems. arXiv:2605.18747. University of Illinois Urbana-Champaign, Meta, Stanford University.