Anthropic Engineering Practice: Harness Design for Long-Running App Development

This is a Chinese forum post summarizing an Anthropic engineering blog article, *"Effective harnesses for long-running development"* by Prithvi Rajasekaran (Anthropic Labs), covering methodology from design-quality evaluation to full-stack code generation.

Key points

Why naive single-agent implementation fails

1. Context anxiety: models lose coherence as the context window fills and may stop early. Compaction (in-place summarization) preserves continuity but doesn't fully solve it; context reset — clearing context and handing over a structured state — does. For Claude Sonnet 4.5, reset is essential for long tasks. 2. Self-evaluation bias: an agent asked to grade its own work tends to be overly optimistic, especially on subjective tasks like design, which lack a binary verification equivalent such as software tests.

Frontend design: making subjective quality scoreable

A GAN-inspired Generator-Evaluator loop (5–15 iterations) scores designs on four criteria:

| Criterion | Weight | |---|---| | Design quality (coherent atmosphere/identity: color, typography, layout) | High | | Originality (evidence of deliberate creative choices) | High | | Craft (typographic hierarchy, spacing, contrast) | Low | | Functionality (usability independent of aesthetics) | Low |

Claude defaults to strong craft/functionality but mediocre design quality and originality, so weighting the latter pushes the model toward greater aesthetic risk. The Evaluator interacts with the live page via Playwright MCP; one example evolved a Dutch art museum site from a conventional landing page into a 3D spatial experience (CSS-perspective checkerboard floor, wall art, portal navigation).

Full-stack: three-agent architecture

| Version | Model | Architecture | Duration | Cost | |---|---|---|---|---| | V1 | Opus 4.5 | Initializer + Coding Agent + context reset | 6 hr | $200 | | V2 | Opus 4.6 | Planner + Generator + Evaluator (continuous session) | 3.8 hr | $125 |

• Planner: expands a 1–4 sentence prompt into a full product spec (16 feature specs + visual design language), keeping high-level design and avoiding premature details that cause cascading errors.
• Generator: implements features in sprints (React + Vite + FastAPI + SQLite), self-assesses, uses Git.
• Evaluator: tests the running app via Playwright (UI, API endpoints, DB state) with hard pass/fail thresholds per criterion.
• Sprint Contract: before coding, Generator proposes what it will build and how to verify it; Evaluator reviews and iterates until both agree on the definition of "done" — bridging user stories and testable implementations.

Case study: solo agent vs full harness

Task: *"create a 2D retro game maker with level editor, sprite editor, entity behavior, and playable test mode"*.

• Solo: 20 min, $9 — wasted layout, rigid workflow, and broken core gameplay (entities displayed but unresponsive to input due to a broken link between entity definitions and the game runtime).
• Full harness: 6 hr, $200 — working game mode, richer features (sprite animation, behavior templates, sound effects, AI-assisted sprite generation, level designer, game export), built-in Claude integration.

Typical bugs the Evaluator caught: a rectangle-fill tool that only placed tiles at drag endpoints; a delete handler whose condition required selection && selectedEntityId when clicks only set one; a FastAPI route-ordering bug where PUT /frames/reorder was shadowed by /{frame_id} (returning 422).

Build to delete

> "Every component in a harness encodes an assumption about what the model can't do on its own, and those assumptions are worth stress testing."

With Opus 4.6's better planning, long-context retrieval, code review, and debugging, the sprint structure was removed; the Planner remains to prevent underscoping and the Evaluator remains because generation still exceeds the model's reliable solo capability. A browser DAW (Web Audio API) built with the simplified harness took 3 hr 50 min at $124.70; QA still caught real gaps across rounds — non-draggable clips, stub-only audio recording, missing synth panels, sliders instead of graphical EQ curves — showing Evaluator value on "last-mile" detail omissions.

Lessons

1. Context reset beats compaction for context-anxious models. 2. Separating Generator and Evaluator is a strong lever — an independent evaluator can be tuned into a skeptic. 3. Subjective quality can be made scoreable via explicit weighted criteria. 4. Sprint Contracts prevent cascading errors and scope drift. 5. Harness complexity should shrink as model capability grows; verify assumptions and delete components that no longer earn their cost.

The post closes by mapping these practices onto the OpenClaw project (context reset strategy, evaluator agents, sprint contracts, build-to-delete reviews, Playwright MCP testing).

Source: https://www.anthropic.com/engineering/harness-design-long-running-apps