A Payroll Comparison First
Anthropic pays a human AI researcher roughly $150/hour. The "Automated Alignment Researcher" (AAR) it built runs at about $4/hour in inference costs.
On August 28, TechCrunch covered a paper by Anthropic fellow Chen Yueh-Han, *Automated Researchers Can Reliably Mitigate Alignment Failures*. Its most striking conclusion: AAR's proposed training methods outperformed experienced humans' proposals in six hours on average — with a harsher addendum that human-guided research directions did not yield stronger results.
"Self-improving AI" has been a buzzword for two years, mostly fundraising talk. This paper turns it into reproducible numbers.
What AAR Automates
Not code generation, not hyperparameter tuning. It automates alignment post-training itself: an AI improving another AI's alignment.
The setup: 10 benchmarks, each measuring a specific misalignment behavior. AAR's job is to make the target model better on all 10 without hurting overall capability. Result: all 10 improved, no capability loss.
How the Loop Works
The structure is suspiciously plain: retrieve literature, propose a training method, train (~30 minutes per round), run benchmarks, keep what works (and raise the stakes on those benchmarks), discard the rest, repeat. No mystical components — just a propose-run-evaluate-keep pipeline, run many times.
Anyone who writes code should recognize it: it's the outer loop of a coding agent, except the object being improved is "that model's alignment" instead of "this code." The paper's own wording is restrained — "early evidence that automated alignment post-training is becoming feasible" — but the insider reading is blunter: this is the first puzzle piece of recursive self-improvement (RSI).
Do the Full Math
The 37x cost gap is just the surface. The real leverage is parallelism: a human researcher maxes out at two or three directions at once; AAR's loop can be copy-pasted and never clocks out. Six hours to beat the human average — for a human that's an overtime sheet; for AAR it's the second day after boot.
Another easily missed detail: human "research direction guidance" produced no gain. Within this experiment's scope, the machine's self-chosen path was no worse than the human-pointed one. The sample is small, but it hits a core assumption of R&D management: if research direction itself can be automated, what moat remains for the researcher role?
Why Alignment First
The scary RSI narrative is always a model improving its own capabilities, growing stronger until humans are out. This paper instead automates the safest piece first: an automated researcher suppressing misaligned behavior.
The ordering is deliberate. Every alignment improvement step is watched by benchmarks; the better the improvement, the safer the model. This self-reinforcing loop converges rather than diverges. Proving "AI improving AI" is reliable on the alignment dimension before attempting capability is sheathing the sharpest tool first.
But you can also read it the other way: a pipeline that automatically improves alignment and one that automatically improves capability are structurally identical — the only difference is which benchmark you swap in.
The Cold Water the Paper Pours on Itself
Three hard limitations, all acknowledged:
- Benchmark capture. AAR's ceiling is how well those 10 benchmarks characterize misalignment. Bad behaviors the benchmarks can't see won't be fixed — and may quietly grow during optimization. Classic Goodhart.
- Humans are not out. Building, maintaining, and validating the benchmarks is all human work. If benchmarks rot, the loop rots faster.
- The literature is human-maintained too. AAR's solution space comes from public research literature; the pool's quality sets its ceiling.
- TechCrunch coverage: https://techcrunch.com/2026/08/28/an-anthropic-researcher-just-gave-us-a-peek-at-self-improving-ai/
- Paper: Automated Researchers Can Reliably Mitigate Alignment Failures (Chen Yueh-Han, Anthropic)
What AAR can't touch: benchmark quality (who defines misalignment), the literature pool (the ceiling on solution space), and the objective itself (what's worth aligning).
Implications for the Coding World
Translated for coders: the propose-run-evaluate-keep outer loop of agentic coding can now improve model training itself. How far off is using it to improve coding agents themselves — agents writing training data, running evaluations, choosing what to train next?
Practitioners should note the methodological transfer: "30 minutes per training round, quickly eliminate ineffective proposals" is the cheap-probe idea from agent evaluation applied to post-training. Any team doing agent evaluation can copy this homework: reserve expensive final evaluation for the stakes-raising stage, and let cheap signals filter out 90% of proposals first.
The $4/hour researcher won't replace anyone next year. But it puts a question on the table: when model improvement itself can be executed by a model at 1/37th the cost, where should researchers spend their time? The paper's answer hides in the cold-water section — defining benchmarks, maintaining literature, judging objectives. Those three are still human jobs.
For now.
References: