The 'Delve Into' Disaster: Why AI's Uniform Style Isn't RLHF's Fault

You can smell the AI accent: "delve into," "in conclusion," endless numbered lists. The popular explanation blames RLHF—human feedback trained the model to please people, so its language became sycophantic too. But a 2026 paper by independent researcher Rohan Mahapatra offers a very different answer, based on systematic measurement of 17 models and 24 linguistic features: the AI style is not caused by RLHF. It emerges earlier—so early that RLHF cannot fix it.

| Item | Detail | |------|--------| | Paper | From Context Shift to Stylistic Collapse: Why Training Objectives Matter More Than Scale | | Author | Rohan Mahapatra (Independent Researcher, Cupertino, CA) | | arXiv ID | 2605.28826 | | Submitted | April 8, 2026 (cs.CL) |

1. Measuring the AI Accent

The paper selects 24 linguistic features in four categories:

• Punctuation: semicolons, dashes, ellipses, colons, parentheses
• Discourse markers: "delve into," "in conclusion," "that being said," "robust," "fundamentally," etc.
• Structural elements: numbered lists, bullet points, Markdown headers
• Tone markers: hedging, apologies, formality

Each model generated 1,000 outputs; feature frequencies were extracted with regex and compared against baselines from 100,000 documents each in the Pile and Dolma corpora.

2. The Numbers

Across 13 evaluated models, on average 83% of features (19.9 of 24) deviated more than 10% from human baselines.

| Feature category | Mean amplification | Peak | |---------|------------|------| | Markdown headers | 16,853% | 209,675% (OLMo-1B-Instruct) | | Bullet points | 3,063% | 13,855% (Claude Haiku) | | Numbered lists | 1,949% | 5,181% (Claude Haiku) | | "In conclusion" | 5,048% | 24,791% (OLMo-1B-Instruct) | | "Delve into" | 3,660% | 17,759% (Llama-3.1-8B base!) | | "However" (sentence-initial) | 332% | 813% (Llama-3.1-8B) |

Meanwhile, systematically suppressed features:

| Feature | Remaining share | |------|---------| | Semicolons | 3.2% | | Ellipses | 15.4% | | Dashes | 18.4% | | Parentheses | 23.2% |

This is not a style preference—it is a systematic redistribution of probability mass.

3. RLHF Is Not the Culprit

The paper runs four paired tests: same base model, base vs. instruct versions. If RLHF drove the deviation, instruct versions should deviate significantly more.

Result: none of the four pairs is statistically significant. All p > 0.25.

Most strikingly, Llama-3.1-8B base deviates *more* (12.39) than its instruct version (10.64). The peak "delve into" amplification appears in the base model—never instruction-tuned. The paper's conclusion: the AI accent is not an alignment problem; it predates alignment. It is baked in during pretraining and generation dynamics.

4. Two Mechanisms

Layer 1 — Context Shift. Training corpora cover everything—fiction, forums, news, papers. But deployment concentrates on expository tasks (explain, summarize, analyze), which naturally carry structural markers. Deployment activates a much narrower conditional slice, P(· | expository).

Layer 2 — Absorbing States. Some features sharply reduce the conditional entropy of subsequent tokens. A Markdown header locks in what comes next (content, another header, or a list); a semicolon constrains nothing. Low-entropy features create self-reinforcing generation loops; high-entropy features do not. Combined—context shift raises trigger probability, absorbing states accumulate amplification—this explains six-digit amplification figures.

5. Gentle Correction Backfires

Adding entropy regularization during pretraining yields a non-linear dose-response:

• λ = 0.1 (negligible): deviation 0.96, close to human baseline, but diversity collapses (distinct-4 only 0.406)
• λ = 1.0 (moderate): deviation worsens to 2.16—240% worse than the unregularized baseline (0.63)
• λ = 5.0 (strong): deviation 0.78, with diversity soaring—distinct-4 at 0.803 (+185%), lexical diversity +194%, repetition -89%

Then the surprise: the λ=5.0 model, trained only on 410M parameters (Pythia-410M), was compared against frontier APIs:

| Model | Distance from human baseline | λ=5.0 improvement | |------|---------------|----------------| | Gemini 2.5 Flash | 6.82 | 96.7% | | GPT-4o-mini | 7.21 | 96.9% | | Claude 3.5 Haiku | 12.29 | 98.2% |

A correctly trained 400M model beats hundred-billion-parameter systems on stylistic naturalness. This is the paper's Control Strength Principle: distributional problems need non-linear solutions—half-hearted correction is more dangerous than none.

6. The Feedback Loop

AI-generated text is entering the internet—tomorrow's training data—carrying systematic style deviations. Next-generation models absorb them; each generation compounds the drift. As the paper puts it, absorbing-state mechanisms mean AI text entering future corpora carries self-reinforcing stylistic machinery that compounds across model generations. The deeper question: if AI reshapes human communication environments, what happens to human writing itself over decades?

7. Honest Limitations

Solid: the 17-model × 24-feature coverage; clean base/instruct paired tests; clean causal structure in the λ experiments; the non-linear control-strength effect backed by data.

Unclear:

• Does λ=5.0 transfer to large models? All regularization experiments used Pythia-410M; perplexity rose from 48.4 to 786.5, and whether that trade holds at 70B+ is an open question.
• Are 24 features enough? Regex matching emphasizes precision over recall; subtler deviations (syntax complexity, rhetorical strategy, metaphor density) are missed.
• Causality unverified. Context shift and absorbing states are best-explanation hypotheses, not causally validated mechanisms.
• English only. Whether Chinese exhibits the same dynamics is unknown.
• No human perceptual validation. Distributional distance may not correspond to readers' subjective "AI flavor" judgments.

8. Conclusion: Accent Isn't Personality, It's a Defect

The paper overturns a widely accepted narrative. The AI accent is not a product of RLHF; it is a byproduct of pretraining-generation dynamics, and alignment training cannot remove it because it operates on a model whose linguistic structure has already been reshaped. All "de-AI-flavor" efforts circling around RLHF are aimed at the wrong stage. The right direction—strong entropy regularization during pretraining—trades perplexity for diversity, and suggests that in an era of scale worship, training objectives matter more than model scale.

References: 1. Mahapatra, "From Context Shift to Stylistic Collapse: Why Training Objectives Matter More Than Scale", arXiv:2605.28826, 2026. 2. Holtzman et al., "The Curious Case of Neural Text Degeneration", ICLR 2020. 3. Kirk et al., "Understanding the Effects of RLHF on LLM Generalisation and Diversity", arXiv:2309.02926, 2023. 4. Mitchell et al., "DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature", ICML 2023. 5. Shumailov et al., "AI Models Collapse When Trained on Recursively Generated Data", Nature 631, 2024.