AutoHarness Technical Deep Dive: Thompson Sampling, Critic Engineering, and Harness Architectures

> A Feynman-style teardown of AutoHarness's gears — a technical supplement to "The AI That Always Wanted to Cheat at Games." If you've read the previous piece, this one takes you under the hood.

---

1. Reframing the Problem: Why LLMs "Know the Rules but Can't Follow Them"

The AutoHarness paper offers a sharper diagnosis than "the LLM knows knights move in L-shapes but won't do it":

The LLM's failure mode is not missing knowledge, but an unreliable mapping from knowledge to execution.

Key data point: in TextArena chess, 78% of Gemini-2.5-Flash's losses came from illegal moves. Yet the model "understands" the rules — it can correctly describe knight movement and castling conditions. The problem occurs at the execution stage: when the board is complex and strategy competes with rule-checking, the model's internal world model "drifts," generating moves that look plausible but violate the rules.

It's like an experienced driver who knows every traffic rule but occasionally runs a red light at a complex intersection while juggling navigation, pedestrians, and signals. Attention resources compete between strategic tasks and rule verification.

AutoHarness's insight: don't leave rule verification to the LLM's internal reasoning — externalize it into executable code.

---

2. Core Architecture: Dataflow Differences Across Three Harness Types

The three variants aren't a simple strictness gradient — they differ fundamentally in control allocation.

2.1 Harness-as-Action-Verifier (default)

• The LLM decides "which move to play" (policy); the Harness decides "whether that move is allowed" (verification)
• Illegal moves are rejected and the LLM regenerates
• The LLM stays inside the control loop; the Harness is a gatekeeper

Key property: general reasoning is preserved. The Harness only intercepts illegal actions and never interferes with legal ones — a brilliant legal move passes through untouched.

2.2 Harness-as-Action-Filter

• The Harness first enumerates all legal actions; the LLM only picks from this pre-filtered set
• The LLM's decision space is pre-pruned by the Harness

Key difference: the Verifier checks *after* generation; the Filter restricts *before*. Filtering is stricter but requires enumerating all legal moves — hard in complex games where legality may require deep search.

2.3 Harness-as-Policy

• No LLM calls at runtime
• The Harness itself contains the full strategy: heuristics, search algorithms, or rule engines
• The LLM generates code at training time; the code runs standalone at test time

This is an extreme form of knowledge distillation — freezing LLM reasoning into deterministic code. The cost is generality; the payoff is zero-cost determinism.

---

3. Thompson Sampling: Exploration–Exploitation in Code Space

3.1 Why not greedy or random search?

The code generation space is combinatorially explosive. A harness may contain dozens of branches, functions, regexes, and state variables. Random mutation wastes most attempts; greedy search gets stuck in local optima.

Thompson Sampling advantages: 1. Maintains a quality distribution (not a point estimate) per candidate 2. Samples from the distribution, naturally balancing exploration and exploitation 3. Distributions narrow as test data accumulates, automatically shifting from exploration to exploitation

3.2 Concrete implementation (reconstructed)

• Each tree node = one harness version: root is the initial template (empty propose_action and is_legal_action); children are parent code plus an LLM-generated mutation
• Node score = Beta distribution: each rollout yields a binary success/failure; model success rate as Beta(α, β) with α = successes + 1, β = failures + 1
• Thompson Sampling: sample a value from each Beta distribution and refine the node with the highest sample

Why Beta? Binary outcomes + conjugate prior = analytic updates, no MCMC; the distribution narrows with more tests; it naturally supports reward-shaped metrics like average legal-move accuracy.

3.3 Convergence

Average of 14.5 iterations to converge, under these assumptions: one child per iteration; each rollout runs in 10 parallel environments up to 1000 steps; convergence when legal-move success rate = 1.0 or timeout.

Hardest games:

• GermanWhist-v0: 43 iterations
• Cryptarithm-v0: 45
• Othello-v0: 62
• Chess-v0: 64

Common trait: complex, state-dependent rules (check, castling, and en passant legality all depend on global state).

---

4. Critic Engineering: Prompt Design for Error Attribution

The paper mentions briefly that the Critic "consolidates error types," but this is a crucial engineering component.

Critic inputs: 1. Current harness code 2. All failed rollouts (state + action + expected + actual result) 3. Environment error messages, if any

Critic output (inferred): structured error attribution, e.g., "is_legal_action false negatives (3 cases) — root cause: no check for check state; propose_action edge-case bug (1 case) — early return when col=0," plus concrete improvement suggestions.

Why not feed raw error logs to the LLM directly? You could, but raw logs are scattered, repetitive, and noisy. The Critic acts like feature engineering — distilling raw test data into error patterns the LLM can use efficiently, analogous to advantage estimation in RL: not "this step scored -1" but "this step is worse than average, and here's why."

Trigger conditions: two error types, two refinement strategies

• Case A: is_legal_action() returns True but the move is actually illegal → refine both functions (the verifier missed it, and the proposer may have offered illegal candidates)
• Case B: is_legal_action() returns False but the move is actually legal → refine only propose_action (a legal move was wrongly filtered)

This distinction avoids blanket retraining and targets the fault point precisely.

---

5. Precise Comparison with Related Work

| Dimension | Code-as-Policies | AlphaEvolve | Eureka | Voyager | Reflexion | AutoHarness | |---|---|---|---|---|---|---| | Generation target | one-shot code | full algorithms | reward functions | executable skills | verbal reflection | Harness / policy | | Search mechanism | none | evolutionary | evolutionary | skill library | verbal RL | Thompson Sampling tree search | | Feedback | none | unit tests | env rewards | execution | self-assessment | env execution + error attribution | | Iteration | one-shot | multi-generation | multi-generation | incremental | single-round | multi-round refinement + convergence criterion | | Runtime | pure code | pure code | LLM + reward | LLM + skills | LLM | LLM + Harness / pure code | | Use case | robot control | algorithm discovery | RL tasks | open-world games | general reasoning | strict-rule environments |

5.1 vs. AlphaEvolve

AlphaEvolve uses the LLM as a mutation operator over codebases. AutoHarness differs in search space (constraint layer/policy vs. full algorithms), evaluation (environment interaction vs. unit tests), and goal (reliable LLM execution vs. new algorithm discovery).

5.2 vs. Code-as-Policies

AutoHarness wraps the LLM rather than replacing it, has a feedback loop, and enforces a hard 100%-legality convergence target.

5.3 vs. Eureka

Eureka generates reward functions verified via expensive RL training; AutoHarness generates the control loop verified by cheap single-run execution.

---

6. Harness-as-Policy: Knowledge Distillation at Its Limit

Harness-as-Policy achieves an average reward of 0.870 across 16 single-player games, beating GPT-5.2-High (0.844) and Gemini-2.5-Pro (0.707).

Not because "code is smarter than LLMs," but:

1. Specialization: a general LLM must cover all knowledge; the policy dedicates all complexity to one game 2. Determinism: LLMs are probabilistic — the same position can yield different moves; in strict-logic games, that randomness is a liability 3. Implicit precomputed search: dozens of refinement iterations amount to a search whose results are frozen into the final code 4. Zero-latency responses: LLM API calls take hundreds of ms to seconds; code executes in microseconds

Limitation: two-player games fail

The paper reports no Harness-as-Policy results for two-player games. Two-player play requires opponent modeling — dynamic game theory beyond static code. The paper cites Lehrach et al. (2025)'s Code World Models (full state-transition functions + MCTS) as a possible future solution.

---

7. Open Questions

7.1 Cross-environment generalization

AutoHarness trains per-game; a chess harness won't work for Go. Future direction: meta-learned harnesses that transfer components (board traversal, bounds checking) to structurally similar games.

7.2 Harness composability

Could harnesses compose — a base harness for generic board-game validation (bounds, turn alternation) plus game-specific harnesses for special rules? Modular design would bring AutoHarness closer to a software "library" concept.

7.3 From games to the real world

The idea extends anywhere there's an explicit, verifiable rule set:
• Code generation: a syntax/type-safety harness alongside generated code
• Robot control: physics-constraint verification (collision, joint limits) before acting
• Database operations: permission harnesses for SQL (no DROP TABLE, no cross-database queries)
• Medical diagnosis: knowledge harnesses for contraindications and dosage limits

8. Conclusion: AutoHarness's Methodological Legacy

AutoHarness's real contribution is a new LLM system design paradigm:

> Split the LLM's capabilities into a "strategy layer" and a "rule layer": keep general reasoning in the LLM, externalize rule enforcement into verifiable code.

This mirrors classic layered architecture: operating systems isolate hardware in the kernel; languages delegate memory management to the runtime; AutoHarness delegates rule checking to the harness. The LLM doesn't need to "remember" every rule — it just needs to know rules exist, and the harness enforces them. This separation improves reliability and frees the LLM's limited context and reasoning budget for genuinely creative work.

That's why "small model + Harness" beats "large model raw": the harness doesn't make the small model smarter — it eliminates the possibility of dumb mistakes, letting its limited intelligence perform reliably.

---

References

1. Lou, X., Lázaro-Gredilla, M., Dedieu, A., Wendelken, C., Lehrach, W., & Murphy, K. P. (2026). AutoHarness: improving LLM agents by automatically synthesizing a code harness. *arXiv preprint arXiv:2603.03329*. https://arxiv.org/abs/2603.03329 2. Liang, J., et al. (2023). Code as Policies: Language Model Programs for Embodied Control. *ICRA 2023*. 3. Novikov, A., et al. (2025). AlphaEvolve: A coding agent for scientific and algorithmic discovery. *arXiv:2506.13131*. 4. Ma, Y. J., et al. (2024). Eureka: Human-level reward design via coding large language models. *ICLR 2024*. 5. Lehrach, W., et al. (2025). Code world models for general game playing. *arXiv:2510.04542*.