Scaling Laws of Wild AI Text: 27.5% of Web Content Is Now AI-Generated, But Its Training Value Depends on How Data-Hungry You Are

A Qualitative Shift Happening Right Now

In June 2026, 27.5% of tokens in Common Crawl—after FineWeb quality filtering—were flagged as AI-generated by Pangram. Two months later, in August, that number rose to 31.1%. Two years ago it was only 10.1%.

This is not synthetic data from a lab, nor deliberately generated training corpora. This is "wild AI text"—content AI wrote for human readers, naturally appearing on the web, mixed into human text, scraped by crawlers, and entering pretraining corpora.

A team from UMass and Google DeepMind asked a sharp question: How does wild AI text affect pretraining? What scaling laws does it follow?

Three Kinds of AI Text, Three Fates

The paper distinguishes three types of AI text:

1. Model collapse data: Degenerated data from recursively training models on their own outputs. Research shows this causes models to gradually lose diversity and eventually collapse.

2. Synthetic data: Deliberately generated training data, carefully designed (e.g., rephrasing, instruction tuning) to boost specific capabilities.

3. Wild AI text: Content AI wrote for human readers—blogs, news, reviews, documentation—naturally appearing on the web and mixed into pretraining corpora.

The first two have been studied. The third is this paper's focus, and it is the data type closest to real pretraining scenarios: it comes from multiple models, has uneven quality, no labels, and is mixed with human text.

800 Models

To answer this question, the team pretrained 800 models, from 19.9M to 973M parameters, adding varying ratios of wild AI tokens to a fixed human corpus (from 0 up to 64 AI tokens per human token).

Evaluation was done on three human-text benchmarks: C4, FineWeb, and Paloma, plus evaluation on AI text.

The key finding condenses to one sentence: AI text helps only "data-hungry" models, and harms "data-rich" ones.

Core Findings: A Threshold Effect

Finding 1: AI text helps in the low-data regime, harms in the high-data regime

When human data is below 10 tokens per parameter (the data-hungry regime), adding AI text lowers loss on human text. But starting from the Chinchilla-optimal 20 tokens per parameter, AI text's benefit disappears, and adding more actually raises loss.

Control experiment: The same amount of fresh human text still lowers loss. In other words, AI text is not "equivalent extra data"—when data is abundant, it is harmful.

Finding 2: AI text passes quality filters more easily

This is a disturbing finding. FineWeb's quality filters remove low-quality content, but AI-generated text passes filtering more easily than human text. This means as the AI-text fraction rises, quality-filtered corpora contain an even higher AI share—filters are inadvertently "purifying" AI text.

Finding 3: AI text does not follow Chinchilla scaling laws

Chinchilla scaling laws describe the power-law relationship between model loss and parameters/data. But models trained on AI text do not follow this law. The paper fits dedicated scaling laws for AI text and finds it has its own parameter set.

Finding 4: The optimal AI fraction depends on data volume

On C4, when human data is scarce (5 tokens per parameter), the optimal AI fraction is 37%. But when data reaches Chinchilla-optimal levels, the optimal AI fraction drops below 1%.

Finding 5: Repeating human data vs. adding AI data

A practical question: if you've already used your human data once, is it better to repeat it or add AI text?

The answer depends on data volume. In the data-hungry regime, adding AI text beats repeating human data. But in the data-rich regime, repeating human data beats adding AI text—though both are worse than fresh human data.

Finding 6: Training on AI text changes model writing style

After training on AI text, a model's generated text drifts stylistically toward AI text's style. This means pretraining data doesn't just teach models knowledge—it also teaches models "how to write". If the AI fraction in pretraining corpora keeps rising, future models may converge stylistically—a potential diversity crisis.

Practical Advice: Treat AI and Human Text as Different Domains

The paper's most important practical recommendation: treat AI text and human text as different data domains, keeping them separate in both training and evaluation.

For training:

• Don't mix AI text into human corpora as "equivalent data"
• If data-hungry, you can add AI text, but control the ratio
• If data-rich, filtering out AI text may be better

For evaluation:

• Evaluate separately on human text and AI text
• Reporting only mixed loss masks AI text's effects

Forecast: AI Text Trends 2026–2032

The paper forecasts the future AI-text share of the web. If current trends continue, by 2032 (the projected point when public human text is exhausted), AI text may occupy the vast majority of the web. This creates a positive feedback loop: models train on AI text → generate more AI text → it enters the next pretraining corpus → ...

This is not model collapse—collapse is recursive training on one's own outputs. This is subtler "wild AI text pollution": text from multiple models across generations, with shifting quality and style distributions.

My Thoughts: A "Uncertainty Principle" of Data Quality

This paper raises a deeper question: when 27.5% of the web is AI-generated, the concept of "web data" itself is losing meaning.

In the past, "web text" = "human text." Now, "web text" = human text + AI text, with the AI share accelerating. The data source we use to train models is being changed by the models we train.

It's a bit like the observer effect in quantum mechanics: the act of measurement changes what is measured. We train models on web data → model-generated content enters the web → web data changes → the next generation of models trains on different data.

One finding deserves particular caution: AI text passes quality filters more easily. Quality filters are not neutral—they favor AI text. This may be because AI text is typically "cleaner," better-formatted, and more "like high-quality text." But "resembling high quality" is not the same as "being high quality"—another instance of the "plausible shell" spectrum: AI text excels at passing quality checks, but passing quality checks is not the same as having quality.

Another point worth pondering: Chinchilla scaling laws fail on AI text. The mathematical tools we've used to plan pretraining budgets may no longer apply in the AI-text era. We need new scaling laws—and this paper is a first step in that direction.

---

Paper link: https://arxiv.org/abs/2609.40295

Code repository: https://github.com/pangramlabs/WildAI

HTML full text: https://arxiv.org/html/2609.40295v1