AI Self-Training Doesn't Flatten Language — It Selectively Kills Deep Syntax, Amazon Paper Finds

Overview

An arXiv paper by Ming Liu (Amazon), arXiv:2605.20602, argues that recursive self-training does not homogeneously 'flatten' language model output — it *restructures* it. Surface-level markers amplify while deep syntactic structures systematically die out. The forum post presents a detailed walkthrough of the paper.

Experimental setup

• Models: GPT-2 (124M), Pythia (410M, 1.4B, 2.8B), and OPT (1.3B) — three architecture families.
• Procedure: 11 generations of self-training (generate → fine-tune → generate), with fixed decoding parameters (top-p=0.95, T=0.9) and fixed prompts.
• Features: 17 pre-registered linguistic features stratified into four depth levels:
• Depth 0 (surface markers): discourse connectives (however, moreover, therefore), hedges (perhaps, maybe), dashes
• Depth 1 (local syntax): regular past tense, sentence-initial conjunctions, coordination
• Depth 2 (clause structure): questions, passives, parentheticals, irregular past tense, relative clauses
• Depth 3 (cross-clause/mood): subjunctive mood

Key findings

Asymmetric collapse, not flattening

After 10 generations (GPT-2):

Amplifying:

• Discourse connectives: +126.2%
• Hedges: +44.2%
• Dashes: +28.6%
• Sentence-initial conjunctions: +19.0%

Dying:

• Question marks: -91.7%
• Colons: -64.8% / semicolons: -64.4%
• Parentheticals: -56.8%
• Passives: -55.5%
• Subjunctives: -52.7%
• Irregular past tense: -52.3%

Average change by depth decreases monotonically: d=0 +24.9%, d=1 -10.0%, d=2 -47.2%, d=3 -52.7%.

The Structural Depth Hypothesis (SDH)

A feature's decay rate is predicted by the syntactic nesting depth required to use it — not by its frequency:

• Depth vs. decay rate: Spearman rho = 0.540 (p < 10^-6)
• Frequency vs. decay rate: rho = 0.225 (marginal)
• Partial correlation controlling for frequency: rho_partial = 0.490 (p < 10^-6)
• Depth explains 29–44% of decay variance; the gradient strengthens with model scale (rho = 0.705 for Pythia-2.8B).

Decisive paired comparisons

• Regular vs. irregular past tense: similar baseline frequencies, but regular (d=1) surged +79.7% while irregular (d=2) collapsed -52.3%.
• Dashes vs. parentheses: both mark asides, but dashes (d=0) rose 28.6% while parentheses (d=2) fell 56.8%.
• Sentence-initial conjunctions vs. mid-sentence coordination: the less constrained form rose 19.0%, the more constrained fell 14.4%.

The surface complexity paradox

Standard complexity proxies all *increase*: dependency tree depth +45.5%, clause nesting +33.4%, average word length +25.3%, type-token ratio +9.8%. Yet this apparent richness is carried by chains of discourse connectives over shallow syntax. Implication: AI-text detectors relying on aggregate complexity metrics will be misled after multiple self-training generations; depth-stratified feature panels are needed instead.

The exclamation-mark 'exception'

Exclamation marks are depth-0 yet collapsed -99.3%. The full SDH equation includes a sampling-dependence term: connectives and hedges have high sampling dependence and sufficient baseline frequency, so a rich-get-richer loop kicks in. Exclamation marks have high sampling dependence but a baseline too low (1.03 per 1,000 tokens) to bootstrap the loop — consistent with the joint depth + sampling-dependence model, not with either factor alone.

Implications

• AI-text detection: the glut of 'however/furthermore' in AI text reflects survival of surface markers, not learned sophistication; aggregate metrics fail under recursive training.
• Training data curation: when retraining on synthetic text, deliberately upweight human text rich in deep syntax — literary fiction, legal and scientific prose.
• Model collapse theory: collapse is structural selection by depth, not uniform degradation — a revision of the standard 'flattening' narrative (e.g., Shumailov et al., *Nature* 2024).

Open questions acknowledged by the author

• Depth-2 features show high internal heterogeneity (not a homogeneous layer).
• Only 10 of 17 features had 95% confidence intervals excluding zero across models.
• English-only experiments; cross-linguistic validity untested.
• Scale ceiling at 2.8B parameters; behavior of modern 70B+ models unknown.
• The causal mechanism (e.g., probability-multiplication penalty on nested structures) is hypothesized but not directly verified.
• Coefficient sizes vary with decoding strategy, though the depth gradient was robust across greedy, ancestral, and narrow-nucleus decoding.

References

1. Liu, M. (2026). *Self-Training Doesn't Flatten Language — It Restructures It: Surface Markers Amplify While Deep Syntax Dies.* arXiv:2605.20602. 2. Shumailov, I. et al. (2024). *AI models collapse when trained on recursively generated data.* Nature, 631, 755-759. 3. Dohmatob, E. et al. (2024). *A Tale of Tails: Model Collapse as a change of scaling laws.* arXiv:2402.07043. 4. Guo, Y. et al. (2024). *The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text.* NAACL 2024 Findings. 5. Kobak, D. et al. (2025). *Delving into ChatGPT usage in scientific writing through excess vocabulary.* arXiv:2505.13478.