Chinese Idioms as Compressed Sensing: A Cognitive and Mathematical Model

This is an English-language presentation of a lengthy Chinese forum post that argues Chinese idioms (chengyu) are a linguistic realization of compressed sensing (CS). Because the source is very long, this version summarizes its structure and findings while preserving its core arguments, math, and evidence.

Key points

• Core thesis: Chinese idioms — especially four-character idioms — implement, at the linguistic level, the three pillars of compressed sensing: sparsity, non-adaptive linear projection, and reconstruction algorithms.
• Sparsity (k << N): Modern Chinese uses roughly N ≈ 10,000 common characters, yet each idiom compresses a rich semantic unit into just k = 4 characters (e.g., "指鹿为马" encodes a historical allusion plus a moral about the abuse of power). Each selected character acts like a nonzero sparse coefficient carrying far more than its literal meaning.
• Non-adaptive projection: The conventionalized four-character grid (with its 2+2 prosodic/syntactic structure) functions like a fixed measurement matrix Φ — it is independent of content, ensures stable, recognizable encoding, and preserves the semantic "skeleton" (analogous to the RIP condition preserving signal energy).
• Reconstruction: Human comprehension mirrors a reconstruction algorithm: the low-dimensional four-character input triggers activation of semantic frames, cultural knowledge, and contextual priors, effectively solving an ill-posed inverse problem y = Φx via an L1-like preference for the simplest, most context-consistent interpretation.

Background: compressed sensing

CS theory (Donoho, Candès, Tao) shows that if a signal is sparse in some basis, it can be sampled far below the Nyquist rate and exactly reconstructed. The basic model is:

\[y = \Phi x + n\]

where x is the original signal, Φ a measurement matrix, and n noise. Sparsity is measured by the L0 norm; since L0 minimization is NP-hard, CS uses the convex relaxation (Basis Pursuit):

\[\min_x \\|x\\|_1 \quad \text{s.t.} \quad y = \Phi x\]

with noisy variants (BPDN/Lasso): \(\min_x \frac{1}{2}\|y - \Phi x\|_2^2 + \lambda \\|x\\|_1\). Measurement matrices must satisfy the Restricted Isometry Property (RIP):

\[(1 - \delta_K) \|x\|_2^2 \leq \|\Phi x\|_2^2 \leq (1 + \delta_K) \|x\|_2^2\]

Random (Gaussian/Bernoulli) matrices satisfy RIP with high probability, making sampling non-adaptive. Greedy algorithms (OMP, CoSaMP) offer faster but weaker-guaranteed alternatives.

Evidence and analysis from the post

Information-theoretic arguments

• Entropy estimates: a uniformly random character carries ~13.29 bits (\(\log_2 10000\)); actual characters follow Zipf's law at roughly 9–10 bits. A four-character idiom carries at most ~40 bits of literal information yet encodes semantic content (allusions, morals, imagery) far exceeding that sum — an "information gain" reflecting compression efficiency.
• Idiom vocabulary shows a power-law/long-tail distribution: in one news-corpus study of 4,900 four-character idioms, 57.49% occurred fewer than 5 times while fewer than 3% occurred more than 50 times. High-frequency core characters (e.g., numerals) act as anchors enabling robust reconstruction from partial cues (e.g., "三…茅庐" → "三顾茅庐").

Cognitive and neural evidence

• ERP studies suggest literal and figurative meanings of Chinese idioms are processed in parallel rather than sequentially, partly due to the four-character format; syntactic structure (e.g., subject-predicate "愚公移山" vs. coordinate "开天辟地") modulates processing load (P600 effects).
• Idiom comprehension activates broad semantic frames (stories, characters, usage exemplars) — the "decompression" step restoring a rich cognitive structure from four characters.
• Context acts like prior information / regularization in reconstruction, biasing interpretation toward the situationally appropriate sense (e.g., "锦上添花" can be positive or faintly derogatory).
• The prefrontal cortex supports the executive demands (inhibition of literal readings, retrieval, pragmatic selection); N200, N400, and P600 components trace successive stages mirroring iterative reconstruction.

Formal model (Section 5 of the source)

• Define a character alphabet Σ with |Σ| = N; an idiom w = (c₁, c₂, c₃, c₄) becomes a 4-sparse binary indicator vector x ∈ ℝᴺ.
• An idiom–character matrix A ∈ ℝ^(M×N) (M = number of idioms) has exactly four nonzeros per row — a highly sparse "dictionary."
• Generation = projection y = Φx through the fixed four-character grid; comprehension = solving the underdetermined system with a sparsest-consistent-solution criterion, analogous to L1 minimization.

Idioms vs. large language models

The post contrasts human semantic compression (understanding-driven, meaning-preserving, contextually flexible) with LLM statistical compression (data-driven co-occurrence modeling). LLMs can map "画蛇添足" to "多此一举" via correlation but lack the structured cultural knowledge network behind the idiom's story and moral, and can be misled by literal readings ("成语幻觉").

Conclusions and proposed future work

Conclusions: (1) idiom formation satisfies the k << N sparsity condition; (2) the fixed four-character structure is a non-adaptive encoding "measurement matrix"; (3) human comprehension is a robust reconstruction process.

Suggested directions: fMRI studies of semantic reconstruction; cross-linguistic comparison with English idioms (e.g., "kick the bucket"); computational models of the encode–decode cycle tested on corpora; and CS-inspired idiom pedagogy.

*Note: This is a speculative interdisciplinary essay; the analogy between idioms and compressed sensing is conceptual, and the author explicitly calls for empirical validation.*