Weighing 2.83 Million Chinese Characters: Most "AI Writing Tells" Are Backwards

Forum topic · 小凯 · 2026-08-26

Summary

An open-source project called lieflat-less-ai-tone measured 26 popular "AI writing tells" against a corpus of 629 articles (300 AI-generated across Claude, DeepSeek, Gemini, GPT, and Kimi; 329 human-written), totaling 2,826,972 Chinese characters, 95,551 sentences, and 45,721 paragraphs. The data overturned most checklist items: human writers use metaphors 2.4 times more often than AI (and standalone metaphor paragraphs 8 times more), rhetorical questions 17 times more, and specific numbers nearly 3 times more. Sentence-length uniformity, once reported as 51x higher in AI text, showed no difference after a segmentation bug was fixed. Only 11 of 26 features discriminated reliably, mostly at discourse and syntactic levels: zero-anaphora paragraph-opening commentary (R=4.4), colon-introduced empty lists (R=9.4), idealized personified metaphors (R=7.3), and contrastive structures (R=3.4). Feature frequencies varied up to 47x across models (dash usage), showing no single "AI style" exists. The team released executable rewriting rules (SKILL.md) with whitelist, information-conservation, and no-subjective-judgment constraints, and documented six measurement errors, four of which would have produced reversed rewriting rules if uncorrected.

A widely circulated checklist in Chinese writing circles claims to identify AI-generated text by traits like excessive dashes, too many metaphors, constant self-Q&A, and suspiciously uniform sentence lengths. The open-source project lieflat-less-ai-tone on GitHub decided to actually measure these claims.

The Corpus

Researchers stacked 629 articles: 300 AI-generated (covering Claude, DeepSeek, Gemini, GPT, and Kimi) and 329 human-written — 2,826,972 Chinese characters, 95,551 sentences, and 45,721 paragraphs in total. They then measured all 26 items on the popular checklist.

Items Measured Backwards

Metaphors. Human writers use metaphor markers 2.4x more frequently per thousand characters, and open paragraphs with metaphors 8x more often — the opposite of the popular claim. The real difference is in the vehicle: AI prefers idealized professional personas ("like a wise mentor," "a tireless reviewer," $$R = 7.3$$ ), while humans write "like an old craftsman" or "like a well-connected doctor friend."

Rhetorical questions. Humans use rhetorical questions 17x more in body text. Following the popular advice to delete them pushes text further from human writing.

Sentence-length uniformity. An early measurement claimed AI uniformity was 51x that of humans, but the segmentation script had failed to filter Markdown tables and punctuation-less list items. After the fix, the ratio was 0.87 — no difference.

> Frequency ratio $$R$$ : feature frequency in AI text divided by frequency in human text. $R \ge 2.0$ means AI is markedly higher; $$R < 0.8$$ means humans are higher.

The 11 Features That Actually Discriminated

Zero-anaphora paragraph-opening commentary ( $$R = 4.4$$ , highest discriminating power). New paragraphs open with judgments like "worth noting" without saying what is being commented on. Total connective density is nearly identical on both sides; only the missing anaphor differs. Adding one "this" fixes it.

Colon-introduced empty lists ( $$R = 9.4$$ ). A sentence carrying no information, ending in a colon, announcing that a list follows. Near-zero on the human side.

Idealized personified metaphors ( $$R = 7.3$$ ). Comparing tools to "a wise mentor" or "a digital secretary" with flattering modifiers. Metaphors per se aren't the problem — only this floating-persona subtype.

Contrastive structures ( $$R = 3.4$$ ). "Not A but B" forced where no contradiction needs correcting.

Specific-number density ( $$R = 0.35$$ , reversed). Humans use concrete numbers nearly 3x more; AI prefers "significantly improved" and "greatly optimized."

The Same Feature Varies 47x Across Models

Dash frequency: DeepSeek 5.16 per thousand characters, Claude 4.25, GPT only 0.11 — a spread of nearly 47x:

\frac{\text{DeepSeek dash rate: 5.16}}{\text{GPT dash rate: 0.11}} \approx 46.9

Prompt-style colons peak in DeepSeek and Claude; question-form headings in Gemini; contrastive structures in GPT. Rules derived from a single model don't transfer — there is no unified "AI style."

From Statistics to Rewriting Rules

The 11 validated features became executable rewriting rules (published as SKILL.md) under three hard constraints:

1. Whitelist principle. Only the 11 listed items are touched; unmatched sentences are kept verbatim; heading levels, paragraph order, and list structure unchanged. 2. Information conservation. Every content word in the rewrite must trace to the original. No new names, numbers, dates, or causal claims; no dropping hedges like "possibly." Changing "may improve" to "improves" is tampering with judgment strength. 3. No subjective semantic judgments. Metaphor aptness, rhetorical-question necessity, and example-pileup assessments are excluded as non-operationalizable.

Six Documented Measurement Failures

All six stemmed from regex operators matching more broadly than their rule definitions. One example: long pre-modifier frequency initially measured 3.04/thousand characters; after excluding ordinary short sentences, it dropped to 0.35. Another: the metaphor-marker lookup table (11 fixed terms) missed all actual metaphor expressions, initially reading 0.000 before the corrected $$R = 0.42$$ . Four of the six errors would have entered the rule set uncorrected, two of them in the wrong direction — rewriting by faulty rules would make text *more* AI-like.

References

Project: *Lieflat Less AI Tone*, by Larashero & Moxt open-source community (2026) — github.com/larashero3-dotcom/lieflat-less-ai-tone
Models covered: Claude claude-opus-4-6, DeepSeek deepseek-v4-pro, Gemini gemini-3.1-pro, GPT gpt-5.6-sol, Kimi kimi-k3
Biber, D. (1988). *Variation across speech and writing*. Cambridge University Press.