Anthropic Engineering Practices: Harness Design for Long-Running AI App Development

Forum topic · 小凯 · 2026-04-04

Summary

An Anthropic engineering blog post by Prithvi Rajasekaran details how the company designs harnesses for long-running AI application development. The approach tackles two core failure modes: context anxiety, solved via full context reset rather than compaction, and self-evaluation bias, solved by separating generation and evaluation into distinct agents. For subjective front-end work, a GAN-inspired Generator-Evaluator loop scores designs on four criteria—design quality, originality, craft, and functionality—turning aesthetics into an optimizable, weighted rubric. For full-stack development, a three-agent architecture (Planner, Generator, Evaluator) uses Sprint Contracts to negotiate acceptance criteria before coding, with Playwright-based QA catching real defects such as broken event handlers, selection logic bugs, and FastAPI route-ordering issues. Comparative runs showed a three-agent harness costing $200 and six hours produced a rich, working 2D game maker, while a $9 solo run yielded broken core functionality. Anthropic's 'build to delete' philosophy stresses simplifying harnesses as model capabilities improve: after Opus 4.6's release, Sprint structures were removed, cutting a DAW build to roughly 3.8 hours and $125 while QA still caught stubbed features.

This is a summary and translation of an Anthropic engineering blog post by Prithvi Rajasekaran on harness design for long-running application development with Claude models.

Original article: https://www.anthropic.com/engineering/harness-design-long-running-apps

Key points

Two core failure modes in long-running agent tasks:
Context anxiety — models lose coherence as the context window fills. Compaction (summarizing earlier dialogue in place) preserves continuity but doesn't eliminate the problem. A full Context Reset with structured handoff works best; for Claude Sonnet 4.5, it is described as necessary for long tasks.
Self-evaluation bias — agents evaluating their own work tend toward over-optimism, praising mediocre output. This is especially damaging for subjective tasks like design, where no binary test equivalent exists.
GAN-inspired Generator-Evaluator for front-end design. A separate Evaluator scores designs over 5–15 iterations using Playwright MCP against the live page, on four weighted criteria:

| Criterion | Weight | |---|---| | Design Quality (cohesive mood, identity) | High | | Originality (deliberate creative choices) | High | | Craft (typography, spacing, harmony) | Low | | Functionality (usability) | Low |

Claude already performs well on Craft and Functionality by default; weighting Design Quality and Originality pushes the model to take greater aesthetic risks. In one case, a Dutch art museum site evolved from a conventional landing page into a 3D spatial experience with CSS-perspective rendering.

Three-agent full-stack architecture (V2: Opus 4.6):
Planner expands a 1–4 sentence prompt into a full product spec (e.g., 16 feature specs plus a visual design language) while avoiding premature detail specification that causes cascading errors.
Generator implements features in sprints (React + Vite + FastAPI + SQLite), with self-assessment and Git version control.
Evaluator tests the running app via Playwright — UI functionality, API endpoints, database state — with hard thresholds per criterion.
Sprint Contracts bridge user stories and testable implementations: before coding, the Generator proposes what it will build and how it will be verified; the Evaluator reviews and both iterate until they agree on the definition of "done."

Solo run vs. full harness

Task: "Create a 2D retro game maker with a level editor, sprite editor, entity behaviors, and playable test mode."

Solo (single agent): 20 min, $9. Wasted layout space, rigid workflow, and — critically — broken core gameplay: entities rendered but unresponsive to input, caused by a broken connection between entity definitions and the game runtime.
Full harness: 6 hr, $200. Additional sprite animation, behavior templates, sound effects, AI-assisted sprite generation, a level designer, and game export. The game mode was fully playable.

Typical defects the Evaluator caught via Sprint Contract criteria:

Rectangle fill tool only placed tiles at drag start/end — fillRectangle existed but wasn't correctly triggered on mouseUp.
Entity deletion required both selection and selectedEntityId, but clicks only set selectedEntityId; the condition needed selection || (selectedEntityId && activeLayer === 'entity').
PUT /frames/reorder was defined after the /{frame_id} route, so FastAPI matched "reorder" as an integer frame_id and returned a 422.

Build to delete

> "Every component in a harness encodes an assumption about what the model can't do on its own, and those assumptions are worth stress testing."

When Opus 4.6 shipped (better planning, long-context retrieval, code review, and debugging), Sprint structures were removed. The Planner was kept to prevent underscoping, and the Evaluator was kept because it still adds value for tasks beyond reliable generator capability. QA moved to a single pass at the end.

A browser-based DAW built with the Web Audio API under this simplified harness took 3 hr 50 min and $124.70 total across three build/QA rounds. QA still caught real gaps: clips couldn't be dragged on the timeline, no synth/drum UI panels, audio recording was stub-only, clip trimming/splitting unimplemented, and effects were numeric sliders rather than graphical EQ curves.

Core takeaways

1. Context Reset > Compaction for context-anxiety-prone models. 2. Generator-Evaluator separation is a strong lever — easier than making a generator critique itself. 3. Make subjective quality scorable via explicitly defined, weighted criteria. 4. Sprint Contracts prevent cascading errors and implementation drift from user intent. 5. Build to delete: harness complexity should shrink as models improve; each component encodes an assumption worth re-testing. 6. Evaluator value boundary: it justifies its cost when tasks exceed the model's reliable capability range — it is not a fixed yes/no gate.