This post from zhichai.net examines the common 'post-proof-of-concept plateau' in AI engineering: LLM-based agents that perform brilliantly in demos frequently stumble in production because they cannot self-diagnose or self-correct failures, leaving human engineers to handle every error manually—a scalability nightmare.
Key points
The core problem
- Traditional AI systems are static and human-dependent. Each failure becomes a ticket for manual analysis, throttling system evolution to the pace of human review.
- The proposed solution is a fast, repeatable self-evolution loop inspired by biological variation–selection–inheritance, formalized as the GEPA framework (Gather, Extract, Process, Assemble).
- Two-layer attention scanning: a fast pass identifies "information anchors" (isotopically labeled compound names like
hyperpolarized [1-13C]pyruvate), followed by deep analysis of high-density paragraphs ranked by information densityID(S) = MI(S,Q) / length(S). - Thought chains: every claim must cite its source paragraph and evidence, making errors traceable (e.g., confusing "aseptic" vs. "sterile" context).
- Chemical-name guardian: every chemical name in the source must appear verbatim in the summary; score = retained / total. Simplifying "[1-13C]pyruvic acid" to "pyruvate" costs points.
- Length controller: targets ~100 words with a ±20% tolerance band and linear decay outside it.
- Semantic twin detector: cosine similarity between source and summary; a summary with correct facts but a distorted causal chain scored only 0.72.
- LLM-as-a-judge: scores comprehensiveness, faithfulness, and technical accuracy on a 0–1 rubric; its feedback feeds the reflection LM.
- OpenAI Evals platform optimizer – fast prototyping (from "please summarize" to a 15-instruction prompt in ~30 minutes with ~20 human feedback items); best for PoC, weak at scale.
- Static metaprompt loop – fully automated; when all four graders flag failures, the metaprompt agent rewrites the prompt (one evolved to ~800 words with 23 detailed rules). Risk: overfitting to grader criteria.
- GEPA (genetic-Pareto) – evolves a prompt population using sampled trajectories, reflection LM analysis, and Pareto-optimal multi-objective trade-offs; strongest generalization on unseen documents, but highest compute cost.
- Dashboard tracing visualizes call chains (
summarization_agent → chemical_grader → length_grader → llm_judge → metaprompt_agent), latency bottlenecks, and full error provenance. A real incident: retention dropped to 73% after an unseen compound "AH111501 sodium salt" appeared; an over-generalized metaprompt instruction was identified and rolled back. - Weekly automated health checks on new documents trigger re-evolution if the composite score falls below 0.85. The system adapted to new FDA continuous-manufacturing labeling requirements within three weeks, versus a two-month manual-update lag.
- A full GEPA evolution cycle can cost ~$500 in API fees; over-evolution can cause degenerate outputs (e.g., name-stuffing that destroys readability).
- Judge bias gets amplified by evolution, motivating "meta-meta-judges" to monitor the evolution process itself.
- The post frames the shift as moving from "writing code" to "cultivating intelligence": humans become gardeners and guardians—providing feedback, setting grading standards, and approving each evolution—while agents handle routine summarization and experts focus on strategic decisions like FDA scientific dialogue.
The GEPA loop (five stages)
1. Baseline Agent – a deliberately minimal starting prompt (e.g., "You are a summarizer. Summarize this text."). 2. Feedback Collection – a dual mechanism combining human expert qualitative insights with LLM-judge quantitative scoring. 3. Evaluation and Scoring – graders convert fuzzy feedback into precise signals. 4. Prompt Optimization – a metaprompt agent analyzes failures and writes improved instructions. 5. Updated Agent – if the new version passes testing, it becomes the new baseline; the cycle repeats.Case study: pharmaceutical CMC document summarization
The framework is stress-tested on FDA regulatory document (CMC) summarization, where a single missing chemical name can derail a drug approval.The four graders
Three evolution strategies
Evolution case: Hyperpolarized Pyruvate (13C) Injection agent
| Generation | Method | Name retention | Length pass | Semantic sim. | Composite | |---|---|---|---|---|---| | 0 | baseline "summarize" | 62% | 58% | 0.71 | 0.64 | | 1 | platform optimizer | 89% | 72% | 0.83 | 0.81 | | 2 | static metaprompt | 91% | 88% | 0.86 | 0.85 | | 3 | GEPA | 97% | 94% | 0.91 | 0.89 |The final agent's ~1,200-word prompt encodes "genes" for format (45–70 words), naming, compression (e.g., "ISO 7, 0.2μm filtering, aseptic process"), context (aseptic vs. terminal sterilization), and self-checking. It produces summaries in ~2 seconds at 95%+ accuracy—work that takes human experts ~20 minutes—and holds an 87% composite score on 30 unseen validation documents.