When Solaris Meets Transformers
Stanisław Lem's *Solaris* describes an intelligent ocean whose cognitive structure humans can observe but never understand. A September 2026 paper, "Xeno-Interpretability: Investigating the Alien Minds of LLMs" by Pierucci et al., makes a bold claim: LLMs may contain Solaris-like structures — the model organizes information in ways that have no corresponding human concepts.
These structures are called "xeno-representations," and studying them is "xeno-interpretability."
Paper link: https://arxiv.org/abs/2609.20408
---
A Neglected Cardinality Argument
The core argument rests on a simple mathematical fact: the space of distinguishable internal states of a model is far larger than the concept space human language can describe.
A network with N neurons at K activation levels has a state space of K^N. Even at GPT-4 scale (N ≈ 7 billion, K = 2), that's 2^7,000,000,000 — vastly more than the atoms in the observable universe.
Not all states are meaningful, but even the subset that is actually used, reproducible, and causally effective likely exceeds the human concept inventory: roughly 10^5–10^6 common words, perhaps 10^7–10^8 with all technical vocabulary.
The cardinality gap means: if the model organizes information via its own concepts, the vast majority of them cannot be accurately translated into human language. Not because we haven't found the right word — but because no such word exists.
This is an empirical, testable claim, and the paper converts the intuition into an operational research program.
---
Separating "Identification" from "Interpretation"
Traditional interpretability assumes every important internal structure has a human counterpart. Xeno-interpretability breaks that assumption by distinguishing two steps:
Step 1: Experimental Identification. A representation can be "located" — it appears reproducibly across inputs, can be geometrically characterized (directions, manifolds, clusters), can be causally manipulated (activating or suppressing it changes outputs), and links to downstream behavior. This step requires no understanding.
Step 2: Semantic Interpretation. Giving the identified representation a human-readable name. This step is optional and may fail — not because we aren't smart enough, but because human concepts have no counterpart.
This separation is the paper's key methodological contribution: we can scientifically establish "there is something here" even when we cannot say "what it is." It parallels how physics studies dark matter — locatable and manipulable through gravitational effects, yet not directly understood.
---
How to Hunt a Xeno-Representation
The paper proposes a four-stage experimental program:
1. Localization. Use sparse autoencoders (SAEs), probes, or activation-difference analysis to find directions or clusters that are structured but do not match known human concepts. Key criterion: reproducibility across inputs, model instances, and training seeds. 2. Causal verification. Activate or suppress the direction and observe downstream behavior changes. This rules out "just noise." 3. Geometric characterization. Is it linear or a manifold? Orthogonal to known directions? How does it evolve across layers? Geometry itself is a form of understanding, even without semantic labels. 4. Semantic attempt (optional). Try to name it. Success means it was just an undiscovered ordinary concept. Repeated independent failures to find a consistent human translation constitute candidate evidence for a xeno-representation.
The program is falsifiable: a candidate failing causal verification in stage 2 is excluded. Only causally effective yet untranslatable structures count.
---
Why This Matters for AI Safety
Current AI safety paradigms map model behavior onto human concepts — "suppress the deception direction," "monitor the harmfulness feature." RLHF and Constitutional AI rest on this mapping. If xeno-representations exist, our safety framework has a systematic blind spot:
- Single-agent safety. Models may be driven by motivations humans cannot name. How do you align with or monitor a goal you cannot describe?
- Multi-agent systems. Multiple LLM agents may exchange information through token patterns that activate shared xeno-representations — coordinating in ways human observers cannot detect, since the audited messages look perfectly benign. Human concept space simply lacks the counterpart. It resembles forward secrecy in cryptography: intercepting the communication doesn't help, because the "decryption key" lies outside your concept space.
Relation to Existing Interpretability Work
Existing interpretability falls into three categories: concept interpretability (human-concept-aligned features, e.g., Anthropic's dictionary learning / SAEs), mechanistic interpretability (circuits and computations, e.g., Neel Nanda / Transformer Circuits), and behavioral interpretability (input-output patterns in deployment). All three implicitly assume important internal structures map to human concepts, computations, or behavior patterns. If the cardinality argument holds, that assumption is wrong — some structures always fall outside human concept space.
The situation resembles physics circa 1900: quantum mechanics revealed that the micro world has no classical counterpart in principle, not merely undiscovered analogies. Xeno-representations may be AI's "quantum effect" — a structural limitation of human concept space, not a failure of cleverness.
---
An Honest Assessment
The paper has limitations:
1. It is primarily programmatic. No confirmed xeno-representation has been found yet; this is a research program, not an experimental report — though defining the problem and falsifiable criteria is itself a first step. 2. The cardinality argument needs stricter grounding. "Large state space" ≠ "large used state space." The actual amount of untranslatable structure in use could be zero or huge — nobody knows. 3. "Untranslatable" is inherently fuzzy. Is it "ten researchers failed to find a word" or a mathematical proof of no counterpart? The paper favors the pragmatic standard, which introduces subjectivity.
Still, the core contribution stands: it is the first systematic proposal that models may contain humanly incomprehensible concepts, with a methodological framework for studying them. Before it, interpretability defaulted to assuming all important structures were understandable; after it, that default no longer holds.
---
Epilogue: Lem's Prophecy
At the end of *Solaris*, Lem writes that every contact with the ocean is not an act of understanding it, but of understanding the limits of our own understanding.
Xeno-interpretability may be such a mirror. Searching for untranslatable structures inside models means measuring the boundary of human concept space. The epistemological question of the AI era may be this: we built a system more complex than ourselves, and understanding presupposes translation into our language. If some things cannot be translated — we need a new way of understanding, one that does not rely on translation. Xeno-interpretability is a first step in that direction.
---
Paper link: https://arxiv.org/abs/2609.20408