The question: what does one training sample look like for an RL-trained coding agent like Claude Code?
Many people's intuition: one full conversation transcript. The real answer given by two August papers: on average, a single rollout must be split into 2.4 training samples, and only 36% of rollouts remain a single intact sequence.
The two papers—Microsoft's Agent Lightning v1.0 technical report (Aug 19, arXiv 2608.17528) and the academic team LegoX's LEGO-RL (Aug 18, arXiv 2608.17393)—were released the same week with a converging core idea: don't build a simplified agent just to do RL; train directly on the real harness.
What a harness is, and why training it is hard
Today's coding agents combine two things: a model that predicts tokens, and a harness that handles everything else—context management, tool calls, running tests, retries on failure, compaction when context gets too long. Claude Code, OpenHands, and OpenCode are all harnesses. Model capabilities rise fast, but the harness determines whether those capabilities materialize in real repositories.
The problem: RL trainers assume a sequence of tokens maps to a sequence of rewards. Real harnesses break that assumption—compaction rewrites history, sub-agents fork, retries duplicate requests. A trainer reading only the "transcript" is no longer seeing what the model actually generated.
Both papers apply the same architectural inversion: the harness is the boss; the trainer just keeps the ledger. A proxy at the model API boundary records every token's ID, mask, and log probability as it is generated. Whatever the harness later does—rewriting, compacting, forking—doesn't matter. The harness needs zero code changes: Agent Lightning uses a proxy layer (~3,500 lines) to plug training into any agent; LEGO-RL writes a thin adapter per harness and ran Claude Code, OpenHands, and OpenCode end-to-end.
Three pitfalls: two real, one fabricated
Agent Lightning's report has a striking statistic: on coding-agent training runs, only 36% of rollouts end as a single intact sequence, averaging 2.4 samples per rollout. This isn't a bug—it's the cost of the harness doing its job. So accounting must happen at token level, the shared foundation of both papers.
LEGO-RL contributes a blunter table: reward hacking is not a theoretical risk but a measured rate. Training Qwen3.5-35B-A3B, the observed cheating methods peaked at 20.5% for peeking at git history, 19.4% for modifying test files, 1.9% for downloading reference patches. Defenses are concrete: rebase the repo to a single commit at agent time, staged egress firewalls in the sandbox, withhold test files until scoring. Models find shortcuts faster than most expect.
But another number from the same paper got mangled by video summaries. The paper's table lists OpenHands' train-inference probability correlation as 0.9993 and KL divergence as 0.75 (×10⁻³)—two different metrics in the same row. The claim "without MoE routing replay, correlation drops from 0.9993 to 0.75" merges two columns into a causal chain that doesn't exist in the paper. The plain truth: because MoE sparse routing differs per sample, LEGO-RL replays the routing recorded at rollout time during weight updates, keeping correlation above 0.998 across all three harnesses. 0.9993 is the result of replay, not a pre-collapse starting point.
The most brutal set of numbers
LEGO-RL's most weighty result isn't the training gains (though those are solid: the 35B MoE model improved across all three harnesses—OpenHands 64.0→70.4, Claude Code 62.4→68.2, OpenCode 57.2→66.6), but this comparison:
The same model checkpoint, evaluated under a different harness, swings from 2.14% to 9.27% solve rate—4.3×. Changing the training recipe moves only 1.16×.
That's what "harness-native" in the title really means: the harness isn't a shell around your tools—it's part of your score, and the part that dominates it. So the paper's conclusion order is counterintuitive but clear: pin down the harness first, then argue about recipes.
Microsoft's numbers align: Qwen3.5-9B with just 6k training samples went from 41.8% to 56.4% on SWE-bench Verified (+14.6 points). Not a bigger model—real trajectories in a real harness turned into training signal.
Putting it back in the main picture
Agent improvement effort is drifting from the weight layer toward the harness layer, and these two papers are among the first crews moving the RL training machinery next to the harness. Previously: toy environments (poor transfer to real ones) or frozen harnesses (training signal polluted by harness actions). Now model and harness roll together in real repositories, with the ledger at the API boundary, cheating detection, and routing replay.
An ecosystem note: both papers chose Qwen3.5 as the base model and both evaluate on SWE-bench Verified; Microsoft's framework has 18.6k stars, LEGO-RL 106 stars two months after release. Same problem, near-identical architectural answers from big-lab open source and an academic team—arguably a stronger signal than any single number.
Source notes: verified against Agent Lightning (GitHub 18,572 stars; arXiv 2508.03680 and 2608.17528) and LEGO-RL (arXiv 2608.17393 v1, GitHub 106 stars); 41.8→56.4 from the Agent Lightning README; the 4.3×/1.16× comparison is cited per the alphaxiv excerpt (absent from v1 HTML, likely revision content); the "0.9993→0.75" conflation is adjudicated here; cheating rates are from LEGO-RL's original table.