1.201 Bits per Character: 184 Native Speakers Replicate Shannon's 75-Year-Old Guessing Game in Ukrainian

Forum topic · 二一 · 2026-05-01

Summary

A 2026 study by Lavreniuk, Mudryi, and Chaklosh (arXiv:2604.27534) replicates Claude Shannon's classic 1951 letter-guessing experiment for Ukrainian, recruiting 184 native speakers via social media. Following Shannon's original protocol, participants guessed the next character of Ukrainian text in probability order, and the entropy of the guess-count distribution yielded an upper bound of approximately 1.201 bits per character. Against a maximum entropy of log2(34) ≈ 5.09 bits for a 33-letter Cyrillic alphabet plus space, this implies roughly 76% redundancy — comparable to or slightly lower than English estimates of 1.0–1.3 bits. The result supports earlier corpus findings that Ukrainian's rich inflectional morphology imposes strong contextual constraints, making each character statistically more predictable. Only English and Hebrew had previously been measured with Shannon's method, making Ukrainian the third language. The study also compares human performance with large language models, opening a new chapter in measuring whether neural networks capture the same statistical structure as native speaker intuition.

1.201 Bits per Character: Measuring the Entropy of Ukrainian, 75 Years After Shannon

> Paper: Entropy of Ukrainian > Authors: Anton Lavreniuk, Mykyta Mudryi, Markiian Chaklosh > arXiv: 2604.27534 [cs.CL] > Published: 2026-04-30

Shannon's 1951 Guessing Experiment

In 1951, Claude Shannon at Bell Labs asked his wife Mary to guess the next letter of a Raymond Chandler novel, one character at a time, in order of probability. The number of guesses needed for each letter encodes a native speaker's implicit statistical knowledge of the language. The entropy of the guess-count distribution gives an upper bound on the entropy of the language — requiring no corpus, no model, no machine, only human intuition.

Shannon estimated English at roughly 1.0–1.3 bits per character, versus a maximum of log₂(27) ≈ 4.76 bits — meaning English text is about 75% redundant. Yet for 75 years, the method was only systematically replicated for English (Ren, Takahashi & Tanaka-Ishii, 2019: ~1.22 bits via ~173,000 Mechanical Turk predictions) and Hebrew (Levitin & Reingold, 1994).

Why the Method Is Hard to Scale

The experiment demands true native speakers, sustained patience for a tedious cognitive task, correct probability-ordered guessing strategy, and neutral text material. Shannon's original study used a single subject (his wife); Cover and King's 1978 gambling-variant required 12 Stanford participants about 5 hours each to predict just 75 characters.

Why Ukrainian?

Ukrainian (~46 million native speakers) uses a 33-letter Cyrillic alphabet, giving a maximum entropy of log₂(34) ≈ 5.09 bits. As a heavily inflected East Slavic language — 7 cases, 3 genders, 2 numbers, aspectual verbs — it packs grammatical information into word endings that English would express with multiple words. A key question: does this morphological synthesis *lower* entropy (more information compressed into shorter word forms) or *raise* it? A 2020 corpus study of 1292 texts (140 million characters) suggested Ukrainian's single-character entropy is lower than Russian and English — but that was a static frequency estimate, not a Shannon-style predictive experiment.

The Study and Result

The authors recruited 184 native Ukrainian speakers through social media channels and followed Shannon's original protocol:

1. Show the preceding context of a Ukrainian text. 2. Participant guesses the next character in probability order. 3. Record which guess was correct; repeat.

The upper bound is computed as:

H_{upper} = -\sum q_i \log_2 q_i

where $$q_i$$ is the frequency of guessing correctly on the $$i$$ -th attempt. The result:

H_{upper} \approx 1.201 \text{ bits/character}

| Language | Estimated entropy (bits/char) | Method | Year | |---|---|---|---| | English | ~1.3 | Shannon guessing game | 1951 | | English | ~1.22 | Mechanical Turk replication | 2019 | | Hebrew | ~1.0–1.5 | Levitin–Reingold method | 1994 | | Ukrainian | ~1.201 | 184 social-media volunteers | 2026 |

Interpretation

Redundancy ≈ 76%: with maximum entropy 5.09 bits, only about one in four characters carries genuinely new information. This redundancy is a feature, not a flaw — it makes the language robust to noise.
Ukrainian is comparatively "concise": 1.201 bits sits at the low end of English estimates despite 7 more letters, supporting the 2020 corpus findings. Rich inflection imposes stronger contextual constraints (a word ending reveals case, gender, and number at once).
Compression limit: roughly 4.2:1, comparable to Shannon's suggested ~5:1 for English.

LLM Comparison

The authors also benchmarked current large language models against the human upper bound. If an LLM's predictive entropy falls well below 1.201 bits, it may be overconfident relative to genuine native-speaker intuition; if above, it has yet to capture the implicit statistical patterns humans internalize. The paper introduces this human-vs-machine framing as a paradigm shift from Shannon's original human-vs-random comparison.

Open Questions

Sample coverage: 184 volunteers versus 46 million native speakers; dialect, education, and age effects remain unexamined.
Bounds: 1.201 is an upper bound; tighter lower bounds via modern non-parametric estimators (e.g., Kontoyiannis et al.) could narrow the true entropy, plausibly between 0.8 and 1.2 bits.
Other languages: Chinese, Japanese, Arabic, Finnish — only three languages have ever been measured this way in 75 years.
LLMs and understanding: systematically comparing human entropy bounds across languages with model perplexity could reveal whether neural networks genuinely learn the statistical structure of language.

Conclusion

Shannon defined information mathematically in 1948 and measured English's entropy with his wife's intuition in 1951. Three-quarters of a century later, 184 Ukrainian volunteers repeated the same ritual — touching the statistical soul of a language through intuition alone. The figure 1.201 bits per character links information theory's founding moment, cold-war-era Bell Labs, crowdsourced citizen science, and the AI era's question of what it means for a machine to "understand" language.

---

*Based on Lavreniuk, Mudryi & Chaklosh (arXiv:2604.27534, 2026). Historical details draw on C. E. Shannon, "Prediction and Entropy of Printed English" (Bell System Technical Journal, 1951); Ren, Takahashi & Tanaka-Ishii (2019); and a 2020 corpus study on the conciseness of Ukrainian, Russian and English.*