Chinese Idioms as Compressed Sensing: A Cognitive and Mathematical Model

This is a long-form interdisciplinary essay originally posted on zhichai.net. Below is a structured English summary of its six sections.

Key points

• Core thesis: Chinese four-character idioms (chengyu) instantiate the three pillars of compressed sensing (CS) theory — sparsity, non-adaptive linear projection, and reconstruction algorithms — at the linguistic and cognitive level.
• Sparsity: An idiom selects only k=4 characters from a character space of N≈10,000, satisfying the k<<N sparsity condition. Each chosen character acts like a non-zero coefficient in a sparse representation, carrying meaning far beyond its literal sense (e.g., "point at a deer, call it a horse" encodes a full historical allusion and political allegory).
• Information-theoretic view: Single-character entropy in real usage is ~9–10 bits (Zipf-distributed); a four-character string carries ≤40 bits yet encodes far richer semantic content — an "information gain" characteristic of efficient compression. Idiom character usage follows a power-law/long-tail distribution (numeric characters are especially frequent), enabling robust partial reconstruction (e.g., guessing "three ... thatched cottage" from partial input).
• Non-adaptive projection: The rigid four-character ("2+2" metrical) structure is a fixed, content-independent "measurement matrix." Conventionalized, pre-agreed meanings mean speakers encode by retrieving a fixed "compressed package," not by generating new structures. Internal syntax and prosody impose structural constraints (ERP studies show higher processing load, e.g., P600 effects, for subject-predicate idioms vs. coordinate ones).
• Reconstruction as comprehension: ERP research suggests Chinese idioms' literal and figurative meanings are processed in parallel. Understanding activates a semantic frame (allusions, characters, usage exemplars). Context functions like a regularization/prior term guiding the reconstruction toward contextually appropriate readings.
• Cognitive science framing: Information compression is presented as a unified principle of perception, memory (gist, schemas), and language acquisition. Neural mechanisms involve prefrontal cortex for inference and ERP components (N200, N400, P600) tracking stages of integration — analogous to iterative reconstruction.
• Contrast with LLMs: Human idiom compression is semantic and understanding-driven; LLM compression is statistical and pattern-based. LLMs may link idioms to near-synonyms via co-occurrence but lack the cultural-historical knowledge network, sometimes producing literal-minded errors ("chest has a bamboo" misreadings).

Formal model proposed

• Define an alphabet Σ of N characters; an idiom w = (c1, c2, c3, c4) is a 4-sparse binary vector x ∈ ℝᴺ.
• An idiom–character matrix A ∈ ℝ^(M×N) (M idioms, each row with exactly 4 non-zeros) serves as a sparse dictionary.
• Generation ≈ projection y = Φx through the fixed four-character "matrix" Φ; comprehension ≈ solving the underdetermined inverse problem y = Φx via an L1-like minimization over candidate interpretations, using semantic frames and context as priors.
• Validation plan: corpus analysis of idiom frequency long-tail distributions (one cited news-corpus study: 57.49% of 4,900 idioms occur fewer than 5 times; <3% exceed 50 occurrences), Gini coefficient / entropy measures, and per-idiom entropy H(w) = −log₂P(w).

Conclusions and future work

The author concludes that chengyu are natural-language compressed-sensing codes: sparse in form, non-adaptively encoded in structure, and cognitively reconstructed in comprehension. Proposed applications include idiom pedagogy (teaching whole semantic frames rather than rote memorization), text compression/summarization in NLP, and semantically richer AI systems. Future directions: fMRI studies of semantic reconstruction, cross-linguistic comparisons with English idioms (e.g., "kick the bucket"), computational simulation of the encode–decode model, and classroom experiments.

*Note: this is an opinion/theory essay; the CS-idiom mapping is an analogy, and its empirical claims (e.g., entropy estimates, ERP findings) are drawn from cited literature rather than new experiments.*