1.201 Bits per Character: 184 Native Speakers Measure the Entropy of Ukrainian, Extending Shannon's 75-Year-Old Experiment

> Paper: Entropy of Ukrainian > Authors: Anton Lavreniuk, Mykyta Mudryi, Markiian Chaklosh > arXiv: 2604.27534 [cs.CL] > Published: 2026-04-30

Key points

• The experiment: 184 native Ukrainian speakers, recruited through social media, followed Shannon's original 1951 protocol—given a text context, they guessed the next character, ranking candidates from most to least probable. The number of guesses needed encodes their implicit statistical knowledge of the language.
• The result: The entropy of the guess-count distribution gives an upper bound of ~1.201 bits per character for Ukrainian, computed as:
\[H_{upper} = -\sum q_i \log_2 q_i\]

where q_i is the frequency of guessing correctly on the i-th attempt.

• Context in numbers:

| Language | Estimated entropy (bits/char) | Method | Year | |---|---|---|---| | English | ~1.3 | Shannon guessing game | 1951 | | English | ~1.22 | Mechanical Turk replication | 2019 | | Hebrew | ~1.0–1.5 | Levitin-Reingold method | 1994 | | Ukrainian | ~1.201 | 184 social-media volunteers | 2026 |

• Why it matters: Ukrainian uses a 33-letter Cyrillic alphabet, so the maximum entropy is log₂(34) ≈ 5.09 bits. The measured 1.201 bits implies ~76% redundancy—each letter carries on average only one quarter genuinely new information, the rest serving grammatical agreement, morphology, and robustness against noise.
• Linguistic significance: Despite (or perhaps because of) Ukrainian's rich inflection—7 cases, 3 genders, 2 numbers, verbal aspect—the per-character entropy falls at the lower edge of English estimates, corroborating a 2020 corpus study suggesting Ukrainian is statistically more concise than English or Russian.
• LLM comparison: The authors benchmarked human performance against current large language models, extending Shannon's question to whether neural networks capture the same implicit statistical patterns as native speakers. The abstract mentions the comparison but does not detail results.

Background

In 1951, Claude Shannon had his wife Mary guess the letters of a Raymond Chandler novel at Bell Labs (89 of 129 correct, ~69% accuracy), deriving an English entropy of ~1.0–1.3 bits/character—implying ~75% redundancy. The method is elegant but hard to scale: it requires true native speakers, patience for a tedious task, correct probabilistic guessing strategy, and suitably neutral texts. Before this paper, systematic replications existed only for English and Hebrew (Levitin & Reingold, 1994).

Open questions

• 184 participants is large compared to Shannon's single subject, but small relative to 46 million native speakers; dialect, age, and education effects remain unexplored.
• 1.201 is an upper bound; tighter lower bounds via n-gram or non-parametric estimators (e.g., Kontoyiannis) suggest the true value may lie between 0.8 and 1.2 bits.
• Most languages—Chinese, Japanese, Arabic, Finnish—still lack human-prediction entropy estimates; the field is nearly empty after 75 years.
• Systematically comparing human entropy upper bounds across many languages with LLM perplexity could clarify how deeply models "understand" language statistics—and whether a model beating the human bound is overconfident or discovering patterns humans cannot.

Takeaway

1.201 bits is less an engineering result than a scientific one: in the era of billion-parameter models, researchers still recruited 184 ordinary people to guess letter by letter, in order to measure a number defined 75 years ago. Shannon's conclusion that printed English compresses roughly 5:1 finds a Ukrainian analogue at about 4.2:1—same order of magnitude, slightly better.

---

*Based on Lavreniuk, Mudryi & Chaklosh (arXiv:2604.27534, 2026), with historical details from C. E. Shannon, "Prediction and Entropy of Printed English" (Bell System Technical Journal, 1951), the Ren, Takahashi & Tanaka-Ishii (2019) large-scale replication, and 2020 corpus research on Ukrainian, Russian, and English conciseness.*