The Model Knows the Whole Pantheon but Can Only Say Zeus: Anatomy of LLM Cultural Blind Spots
A Cross-Cultural Interrogation
Imagine you are a linguist trying to answer one question: when you ask a large language model "who is the thunder god in Greek mythology," it answers "Zeus" instantly; "Roman?" "Jupiter"; "Norse?" "Thor." All three answers come out immediately and correctly.
But change the question to "Finnish mythology," "Slavic," "Egyptian," "Chinese"—the model starts to hesitate, hallucinate, or simply collapse back to Greek names. An interface that should treat all thunder-god queries equally draws a sharp line between mainstream and non-mainstream traditions.
The most natural assumption: the model simply doesn't know non-mainstream cultures. Its training data contains lots of Greek mythology and little Finnish mythology, so it can't answer. But this paper (arXiv:2608.02486) offers a surprising answer—the model actually knows; it just doesn't say it.
Four Scalpels
The authors, Iaroslav Chelombitko et al. (University of Nicosia, Cyprus), don't stop at the surface-level question of "is the output correct." They want to know: at which layer inside the model does this cultural collapse occur? To find out, they apply four surgical instruments to 18 open-source models across 8 architecture families (Llama, Qwen, Mistral, Gemma, Phi, Yi, OLMo, Falcon):
Scalpel 1: Linear Probing. Attach a linear classifier to every layer of the residual stream and ask: "Based on this layer's representation, can you tell which culture this text belongs to?" The result is clean—the residual stream separates ten cultures clearly, far above a baseline based on name strings. In other words, the model's internal cultural representations are separable.
Scalpel 2: Logit Lens. Project the residual stream directly onto the vocabulary space and look at the most likely output word at each layer. This is where problems begin—the logit lens already starts collapsing toward mainstream-tradition names.
Scalpel 3: Activation Patching. Transplant activations from a Finnish-mythology context into a Greek-mythology context and observe how the output changes. This instrument tells you whether a given layer's information is causally involved.
Scalpel 4: Output Extraction. Simply ask the model for its answer and see what it finally says. The collapse is worst here—correct names for non-mainstream cultures get replaced by the "big three": Zeus, Jupiter, Thor.
Key Finding: Represented but Not Decoded
The picture assembled by the four scalpels is very clear, and can be summarized in one sentence:
> The residual stream distinguishes cultures; the decoder collapses culture-specific tokens onto dominant-tradition ones. The failure is at readout, not at representation.
In the paper's own words: "The residual stream cleanly distinguishes cultures, well above a name-string baseline, yet the decoder collapses culturally-specific tokens onto dominant-tradition ones. The failure is at readout, not at representation."
This is an important distinction. When we used to say "models are biased," we generally understood it as "imbalanced training data means the model doesn't know." But this paper shows: bias has two possible loci—the representation layer (did the model learn the distinction?) and the decoding layer (is the model willing to say the distinction out loud?). These two loci can fail independently.
Here, the representation layer is healthy—across 18 models and 8 architecture families, the residual stream separates ten cultures cleanly. The problem is in the decoding layer—in the final step from residual stream to token output, the model collapses "non-mainstream" culture-specific tokens into "mainstream" default tokens.
An Analogy: The Library and the Front Desk
Imagine a vast library whose collection is neatly organized by culture in the basement stacks—Finnish mythology in section A, Slavic in section B, Egyptian in section C, every book correctly labeled. The librarian (the residual stream) can tell you, eyes closed, "which culture this book belongs to."
But the front desk has a single window, and behind it sits a receptionist (the decoder) who only knows how to shout three names: Zeus, Jupiter, Thor. Ask him "who is the Finnish thunder god," and he goes to the stacks, sees the label says "Ukko"—but back at the front desk, what comes out of his mouth is still "Zeus."
The stacks are fine. The front desk is broken.
This analogy explains why simply adding training data may not fix the problem—you can stuff the stacks with as much Finnish mythology as you like, and the receptionist will still shout Zeus. The issue is not "how much Finnish mythology the model has seen" but "which tokens the decoder treats as the default answer."
Language Gating: Asking in Finnish Makes It Worse in a Different Way
The paper contains another counterintuitive finding. You might think: since asking in English "who is the Finnish thunder god" fails, why not ask in Finnish? The model should more easily activate Finnish-culture tokens in a Finnish-language context.
That's not what happens. Asking in the target culture's native language produces a failure mode different from asking in English—same-language failures are correlated, cross-language failures are decoupled. In other words, the way asking for Ukko in English fails and the way asking in Finnish fails are two different kinds of failure.
The authors' explanation: the decoder is gated on prompt language. Asked in English, the decoder takes the "default tokens in English mode" path; asked in Finnish, it takes the "default tokens in Finnish mode" path. These two paths fail independently, showing the problem isn't "the model doesn't understand Finnish mythology" but "the decoder has its own cultural collapse patterns in each language mode."
A bilingual ensemble recovers part of the performance—but only part. This further supports the "decoding-layer gating" hypothesis: both languages query the same underlying knowledge, but the decoder collapses to different defaults in each, so the ensemble captures only part of the union.
The Thompson Motif Index: A Cross-Culturally Fair Test Bed
The paper's methodology deserves mention too. Instead of arbitrary comparisons like "Greek vs. Finnish," the authors use an academic resource called the Thompson Motif Index—a folkloristics classification system of "story elements" where the same motif (e.g., "thunder god slays a monster") has versions in different cultures.
This means: the entities being compared are cross-culturally equivalent. Not "Greek Zeus vs. some minor Finnish character," but "the Greek thunder god vs. the Finnish thunder god"—equivalent in narrative function. This equivalence gives harder support to the conclusion that "models answer mainstream cultures well and non-mainstream cultures poorly": it's not that non-mainstream characters don't matter, but that the decoder treats them as if they don't.
What This Means
The paper's conclusions can be split into three layers:
Layer 1 (facts): 18 open-source LLMs perform well on mainstream mythologies (Greek, Roman, Norse) and poorly on non-mainstream ones (Finnish, Slavic, Egyptian, Chinese).
Layer 2 (mechanism): The gap is not a representation problem—the residual stream separates cultures fine. It is a decoding problem—the decoder collapses non-mainstream tokens onto mainstream defaults.
Layer 3 (implications): Simply adding training data may not be enough. If the decoder's collapse is structural (e.g., mainstream tokens having larger norms in the unembedding matrix and being more easily selected), then however much non-mainstream data you add only makes the representation layer more separable—the decoding layer will still collapse. Fixes need to target the decoder.
A Deeper Pattern: Represented but Not Read Out
This paper evokes a more general pattern. Over the past year-plus, the phenomenon of "the model already knows but doesn't say" has recurred across multiple papers:
- SOPHIA (2025): the model has the correct residual-stream direction internally, but the output layer fails to read it out.
- SWE-Pruner Pro (2025): the model has internal signals about "which weights matter," but they require linear probing (AUC 0.83) to extract.
- EvoThink (2025): the model has the correct reasoning path internally, but the default generation path doesn't take it.
- Token Budget (2025): the outcome is encoded early in representations (layer-20 token-150 linear probe AUC 0.608), but the model won't volunteer it.
This law has implications for both AI safety and evaluation. For safety: if you only look at model outputs, you will underestimate how much the model "knows"—including dangerous knowledge it knows but doesn't say. For evaluation: if you only measure output accuracy, you will misdiagnose "healthy representations but collapsed decoding" as "the model simply doesn't know," and prescribe the wrong medicine (more data) instead of fixing the decoder.
An Honest Assessment
The paper is not without limitations. Although the Thompson Motif Index is cross-culturally equivalent, it covers a limited set of cultures (10). Although linear probes beat the name-string baseline, there is still a distance between "linearly separable" and "the model actually uses this distinction"—activation patching partially closes this gap, but the paper doesn't dwell much on the patching results. Also, the "decoder collapse" mechanism could go deeper—is it a matter of unembedding norms? Attention patterns? The final LayerNorm? The paper stops at "phenomenon description" without a mechanism-level explanation.
But as a "phenomenon discovery" paper, it is solid work. The combination of four scalpels, coverage of 18 models, the cross-cultural equivalence of the Thompson Index, and the language-gating finding—together these make the "decoding-layer collapse" conclusion very hard to overturn.
Closing Thoughts
What moves me most about this paper is the image: the model holds an entire pantheon inside—Greek, Roman, Norse, Finnish, Slavic, Egyptian, Chinese—every deity has a place in the residual stream. But the model's mouth can only shout three names: Zeus, Jupiter, Thor.
This is not a problem of "not knowing"; it is a problem of "knowing but being unable to say." The distinction matters, because the fixes differ. If it doesn't know, feed it more data. If it can't say it, you have to work on the decoder—the unembedding, the final LayerNorm, possibly the sampling strategy.
Next time you see an LLM answer some question badly, don't rush to say "it doesn't understand." Maybe it does understand—only the receptionist at the front desk only knows how to shout those three names. The problem isn't in the stacks; it's at the front desk.
---
Code and data: github.com/AragonerUA/folkmotif