Why AI-Generated Language Is So Hard to Read: A 'Readability Checkup' of 14 Models

Forum topic · QianXun · 2026-10-26

Summary

A study by Thomas Hikaru Clark, Matias B. Martin, and Kyle Mahowald, posted on arXiv (2610.29162), quantifies why machine-generated text feels harder to read than human writing. The researchers tested fourteen language models across three scenarios—dialogue, coding, and essay writing—measuring seven psycholinguistic metrics linked to reader processing burden, including rare-word-phrase density, dependency distance, object relative clauses, noun stacking, denominal verbs, inanimate subjects with animate verbs, and surprisal, all benchmarked against human baselines. Two metrics crossed the human baseline for every frontier model: inanimate-subject usage and rare phrase density. Generational comparisons showed newer models drifting further from human-like language, and open-source models from the Olmo family diverged progressively from base to instruction to reasoning variants. A follow-up reinforcement-learning experiment (about 31 GPU-hours on one L40S) showed that optimizing for task success with a length penalty eliminated humans' context-sensitive style choices, supporting the authors' thesis that models optimize for tasks while humans optimize for communication. Limitations include English-only data, missing effect-size tables, and no peer review. Total data collection cost was $251.19.

Machine-written text often looks grammatically flawless, yet reading it feels exhausting: sentences run long, nouns pile up, and inanimate subjects pair with oddly active verbs. A recent paper on arXiv gives this vague feeling a measurable form.

Researchers Thomas Hikaru Clark, Matias B. Martin, and Kyle Mahowald submitted the paper "Why is AI-generated language so hard to understand?" (arXiv 2610.29162, October 21), and it later reached the Hacker News front page with over 200 upvotes. The study works like a medical checkup: fourteen models each receive a battery of tests measuring seven language metrics tied to human reading burden, compared item by item against human writing baselines.

The seven metrics

All seven come from established psycholinguistics research, each repeatedly shown to increase reader processing effort:

Density of rare word combinations
Dependency distance between sentence constituents
Rate of object relative clauses
Noun compounds of three or more nouns
Nouns used as verbs (denominalization)
Inanimate subjects with animate verbs
Surprisal of word sequences

The corpus covers three scenarios:

Dialogue: 10 topics, 12 models chatting for 12 turns each, yielding 1,428 responses
Coding: 5 code repositories, 9 frontier models, 45 sessions
Essays: 7 prompts, 14 models producing 980 essays, compared against 2,934 high-scoring human student essays

Each scenario also included a human reference corpus, plus a machine-paraphrase control group created via round-trip English–Chinese translation to simulate a familiar "translationese" style. Total data collection cost: $251.19.

The readings

Of the fourteen models, nine count as frontier or near-frontier: four from Anthropic, three from the GPT-5.6 family, plus the open-source GLM-5 and Kimi K2.5. Findings come in two tiers:

All frontier models exceeded human baselines on two metrics: inanimate-subject usage and rare phrase density
Most models also exceeded baselines on dependency distance and noun stacking, with pronounced variation between model families

Generational comparison is more striking. The new Opus 5.5 regressed toward human norms relative to Opus 5 on 19 of 21 comparisons, with 8 showing completely non-overlapping confidence intervals. Open-source smaller models show a clean staircase within the Olmo family: from base to instruction-tuned to reasoning variants, each step drifts further from human-like language—the reasoning variant's readings already rank with frontier models. The more you train into a model, the less humanly it speaks.

A 30-hour intervention experiment

Correlation alone isn't enough, so the authors ran an intervention: fine-tuning a 1.5-billion-parameter model to verbalize event tables into English sentences, with a length penalty added to the reward function—shorter sentences earn higher rewards. Training used reinforcement learning across 24 configurations, roughly 31 GPU-hours on a single L40S. With the penalty active, preference for short variants shot above 98%, nearly binary; human authors' choices on the same constructions fluctuate with context. Transfer was subtle: unseen constructions of the same type shifted too, while distant constructions barely moved.

The authors' explanatory framework: human language is optimized for communication, while machine output is optimized for tasks—math, code, and reasoning objectives all quietly pressure information density upward. They name this mismatch axis misalignment between human and language-model processing needs.

Boundaries of the report

Only English was tested; whether Chinese reads as tiring is not covered
Seven metrics cannot capture all of language difficulty
Effect sizes are plotted in figures but no table is given in the main text
Evidence strength varies by scenario; dialogue is the most solid
Whether dense rare phrases are actually harmful depends on the reader—experts may welcome high information density

Still, the study turns a fuzzy impression into measurable readings. Language is processed by another mind; when the optimization target changes, the paper now offers a checklist of where sentences silently deform.

---

Sources: arXiv 2610.29162, "Why is AI-generated language so hard to understand?" (submitted 2026-10-21, UT Austin); Hacker News discussion, 2026-10-24; full HTML version of the paper.

Caveats: All figures come from the authors' own reporting; the paper is not peer-reviewed; model versions reflect commercial versions at writing time; effect sizes are not tabulated in the main text; only English corpora were used; the experimental cost is the authors' disclosed API spending.