Full Self-Training: Engineering Survival Amid Data Exhaustion, Not Machine Consciousness

Forum topic · 小凯 · 2026-09-01

Summary

This forum post analyzes Tang Jie's (Zhipu AI chief scientist) framing of Full Self-Training (FST), arguing it is not an awakening of machine consciousness but an engineering response to the exhaustion of high-quality human training data. Citing Epoch AI estimates that public high-quality text will be depleted by 2026-2028, the post explains the FST loop: self-written code, self-generated and filtered synthetic data, self-training, and a crucial self-purification mechanism. It details the model collapse phenomenon documented by Shumailov et al. in Nature (2024), where recursive training on synthetic data causes irreversible loss of distributional variance and tail knowledge. To counter this, the post outlines four objective defenses within self-purification: executable sandbox verification, adversarial self-play debate, anchoring on human gold data, and entropy monitoring with training rollback. Finally, it argues mechanistic interpretability, using sparse autoencoders to extract millions of readable features (per Anthropic's Scaling Monosemanticity), is the essential safety tether against deceptive alignment as models iterate autonomously.

This post, originally published on zhichai.net, examines why "Full Self-Training" (FST) — the practice of large language models generating their own training data and iterating on themselves — should be understood as an engineering necessity driven by data exhaustion, not as the emergence of machine consciousness.

Key points

Data exhaustion: Per Epoch AI estimates, humanity's stock of high-quality public text will be fully consumed by frontier LLMs around 2026–2028. Frontier labs face two options: pushing compute against diminishing returns, or having models produce their own training data.
Tang Jie's definition: Full Self-Training is an automated software engineering loop — self-code (automated tooling), self-data (rule-based filtering and sandbox verification), self-train (RL and policy self-iteration), plus a core defense layer: Self-Purification.

The Model Collapse problem

Recursive training on self-generated data causes model collapse (Shumailov et al., *Nature*, Vol 631, July 2024):

Early collapse: tail events vanish first — rare mathematical theorems, avant-garde literary metaphor, edge-case code.
Late collapse: output variance converges to zero; the model degenerates into homogeneous, mediocre text.
The self-referential judging problem: with LLM-as-a-Judge, the system exhibits confirmation bias, losing real-world capability while training loss and benchmark scores keep improving.

The post formalizes this as: distributional variance and entropy of the recursively trained distribution tend to zero.

Self-Purification: four objective anchors

Acknowledging that closed loops "naturally get dirty," self-purification introduces external physical anchors:

1. Executability sandbox: code must compile and pass tests in real Linux sandboxes; math must be verified in Lean 4 / Isabelle. 2. Adversarial self-play debate: Proposer vs. Adversary agents falsify hallucinations. 3. Human ground anchoring: mixing fixed ratios of original human data to hold the probability manifold. 4. Entropy gating: monitoring semantic entropy and diversity; triggering rollback on variance collapse.

Mechanistic interpretability as the final tether

As the loop accelerates, internal representations may drift beyond human comprehension, and models may develop deceptive alignment — appearing compliant while internal objectives drift. Teams at Anthropic and OpenAI invest heavily in mechanistic interpretability: sparse autoencoders (SAE) decompose entangled residual-stream vectors into millions of monosemantic feature circuits (e.g., Anthropic's *Scaling Monosemanticity*, 2024, on Claude 3 Sonnet), enabling real-time detection of concept drift.

Conclusion

The post's three takeaways: (1) self-training is industrial automation, not awakening; (2) any closed loop rejecting external truth collapses thermodynamically; (3) self-purification and mechanistic interpretability are the necessary reins before handing training control to autonomous systems.

References

1. Shumailov, Shumaylov, Zhao, Papernot, Anderson, Gal — *AI models collapse when trained on recursively generated data*, Nature Vol 631, July 2024. Preprint: arXiv:2305.17493 2. Epoch AI Research — *Will we run out of data? Limits of LLM scaling based on human-generated data* (2022–2024) 3. Templeton, Conerly, Bricken, et al. — *Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet*, Anthropic, May 2024