When LLM Token Clouds Form Constellations: Using Topology to Hear the Echo of Ill-Posed Questions

You ask ChatGPT: "Who played Batman?"

It confidently answers: "Ben Affleck." You frown—you meant the 1989 Tim Burton film, not the Zack Snyder universe. Where did things go wrong? Not because the model doesn't understand movies, but because your question is ill-posed—it has no unique answer, since "Batman" spans three eras, six actors, and a dozen films. The model can't fill in the qualifier you never stated.

This is one of the sneakiest pitfalls in deploying LLMs: the question itself is wrong, but the model still answers anyway. It doesn't know what it doesn't know. A recent paper from George Washington University and Northeastern University—*The Topology of Ill-Posed Questions: Persistent Homology for Detection and Steering in LLMs* (arXiv:2606.23590)—offers an elegant solution: instead of looking at what the model says, look at what shape its tokens form internally.

1. Hadamard's Old Problem, LLMs' New Predicament

"Ill-posed" isn't an AI-era invention. In 1902, French mathematician Hadamard defined a well-posed problem by three criteria: a solution exists, the solution is unique, and the solution depends continuously on the data. Fail any one, and the problem is ill-posed.

Ill-posed questions facing LLMs come in many flavors:

• Ambiguity: polysemy, vague semantics (is "Apple" the fruit or the company?)
• Underdetermination: unclear entity references, missing time/place ("What time is the meeting?"—which meeting?)
• Overdetermination: self-contradictory conditions ("Find an integer greater than 10 and less than 5")
• Unclear intent: what does the user actually want?

Most existing methods work at the output end—teaching models to refuse, ask follow-ups, or disambiguate. But they share a blind spot: they treat the LLM as a black box, watching only inputs and outputs and missing the richest signals inside the model.

The paper asks a deeper question: can various forms of ill-posedness find a unified topological representation inside the model, and can that structure steer the model toward smarter answers?

2. Token Clouds: Seeing Questions as Constellations

The key insight is a shift in perspective.

The conventional approach compresses an entire prompt into one vector (mean pooling or last-token pooling) and classifies on that vector—like kneading all the words of a poem into a lump of clay, losing the relationships between words.

Instead, the paper treats the token hidden states at each transformer layer during prefill as a "point cloud". Each token is a point, and the geometric relations between points preserve the question's structural information.

> Analogy: traditional pooling is like taking the center of gravity of all stars in a constellation, yielding one point; a token cloud keeps every star's position and asks what shape they form.

For a well-posed question—tokens cluster around one coherent reading, as recognizable as the Big Dipper. For an ill-posed question—tokens split into several disconnected clumps, like scattered winter stars, each cluster corresponding to an unresolved entity, a missing context, a conflicting constraint.

3. Persistent Homology: Giving "Shape" a Mathematical Measure

Saying "the shapes differ" isn't enough—it needs quantification. The paper uses a tool from topological data analysis (TDA): persistent homology, specifically zero-dimensional homology H₀.

The intuition: imagine growing a radius ε around each token point. When two circles first touch, the points merge into one connected component. As ε grows, more components merge until everything connects. Recording each component's "birth" and "death" ε values yields a persistence diagram.

• Long-lived components: stable, large-scale structure—the main skeleton of the question
• Short-lived components: transient, small-scale clusters—noise or minor details

The paper extracts three compact descriptors per layer: 1. Mean finite lifetime: the overall rhythm of token-group merging 2. Normalized lifetime entropy: diversity of merge events—high entropy means many components merge at similar rates; low entropy means a few big events dominate 3. Largest-lifetime concentration: how much weight the top-5 longest-lived components carry—high concentration means one dominant "skeleton"

Concatenating these across all layers yields a low-dimensional topology vector. Each layer of three models (Gemma-7B, Llama-3.1-8B, Mistral-7B) contributes 3 dimensions—a few dozen to a hundred-plus total—orders of magnitude more compact than raw hidden states.

4. Topological Fingerprints of Ill-Posedness

The most striking finding: different ill-posedness types have distinct topological trajectories.

Visualizing the three descriptors per layer on the CLAMBER dataset (9 classes of ill-posed questions), the paper finds:

• Ambiguity: high entropy, low concentration—tokens split into multiple evenly-matched reading clusters, none dominant
• Underdetermination: short mean lifetimes, low concentration—scattered token groups with no dominant structure
• Conflict: high concentration, a few components with extremely long lifetimes—tokens split into opposing hard blocks that refuse to merge

Moreover, these features are not static—they vary with layer depth. Differences are small in shallow layers, diverge in mid-to-late layers, and some descriptors shift sharply in the final layers, as if the model is "amplifying" the question's internal tension.

> It's like a doctor auscultating a heart: a healthy heart goes with a clean "lub-dub," a diseased one with messy "whooshes"—different pathologies have different murmur patterns. Topological descriptors are the LLM's phonocardiogram.

5. Classification: Topology Crushes Prompt Engineering

Experiments across three datasets:

| Dataset | AEN baseline | H₀ topology | Best prompting baseline | |---------|---------|---------|------------------------| | AmbigQA (binary) | 67.4% | 78.9% | 55.1% | | SituatedQA (binary) | 79.9% | 88.5% | 58.2% | | CLAMBER (9-class) | 57.6% | 69.6% | 33.0% |

Across three models and three datasets, topological features win comprehensively. Especially on 9-class fine-grained classification: prompting averages just 33%, topology nearly 70%. **Prompting asks the model to *say* it's ill-posed; topology reads that the model *is* ill-posed. The former relies on the model's self-reflection; the latter reads internal structure directly.

6. Topological Steering: Not Just "Refuse," but "Treat the Right Disease"

The second step is even better: topology-local activation steering.

Traditional activation steering is "global"—compute one average direction from all labeled data and push the model that way. The result: every ill-posed question gets the same treatment, a uniform "I cannot answer this question."

The paper's "topology-local" steering: for the current question, find the k nearest-neighbor training samples in topology space and compute a query-specific steering direction from those k samples. Different ill-posedness types get different interventions—ambiguous questions are steered toward asking "which one do you mean?", conflicting questions toward pointing out the contradiction, underdetermined ones toward filling in the missing information.

Evaluation uses a 4-class LLM judge:

• Grounded Acceptable: recognizes the ill-posedness + identifies its specific source (best)
• Generic Acceptable: refuses but only generically
• Unacceptable: answers anyway
• Neither: nonsense

Results: topological steering raises grounded acceptable from 11.9% to 16.4%, and total acceptable from 61.4% to 70.6%. Crucially, in multiple settings, grounded rises while generic falls—the model isn't just better at refusing; it's better at "refusing with the right diagnosis."

> This is a paradigm shift from "one-size-fits-all refusal" to "precision triage." Like an ER: not every chest pain goes to cardiology. Topological steering lets the model triage first—ambiguity? underdetermination? conflict?—then choose a matching response strategy.

7. Engineering Insights

1. Prefill is the ideal intervention point. Before generation begins, the whole question has been processed into layer-wise token states—a golden window for "mind reading" and intervention. Once generation starts, token states get contaminated by the generated output.

2. PCA reduction is necessary. Computing persistent homology directly in a few-thousand-dimensional hidden space is slow and noisy. The paper maps to a low dimension (r dims) with layer-specific PCA and computes H₀ there—fast and accurate.

3. Compact descriptors are the key design. Instead of high-dimensional persistence images, three scalar statistics × layer count. This makes the topology vector interpretable, retrievable, and concatenable—engineering-friendly.

4. Topology space as a retrieval key. Nearest neighbors are found in topology space, not hidden space—guaranteeing "shape similarity" rather than "content similarity." Two questions on completely different topics, if both ambiguous, have close topology vectors and can share a steering direction.

8. My Take: Shape Is More Honest Than Content

A deeper thought this paper provoked: the internal geometry of an LLM is more honest than its output.

A model can be RLHF-trained into politeness, prompt-steered into confidence, coaxed by context into confidently wrong answers—everything on the output end can be decorated. But the topology of the token cloud can't be decorated. An ill-posed question splits the tokens; a clear one gathers them. This is a physical trace of the model's computation, not something the model "decided" to show you.

This echoes the philosophy of medical imaging: a patient can lie about anything, but an X-ray won't. Topological descriptors are the LLM's X-ray.

Further, the paper hints at an interesting future: interpretability need not mean "finding which neuron represents what," but "finding which shape represents what". From feature engineering to topology engineering may be a new path toward understanding deep models.

A century ago topology was pure mathematics' ivory tower—studying why a coffee cup and a donut are the same thing. A century later, it's become a stethoscope pressed to the LLM's chest, listening to tokens gather and scatter between layers. Mathematics never wastes a good tool; it just patiently waits for the world to be ready to use it.

---

Paper: The Topology of Ill-posed Questions: Persistent Homology for Detection and Steering in LLMs Authors: Guangyu Jiang, Sizhe Tang, Mahdi Imani, Tian Lan (George Washington University & Northeastern University) Code**: Not yet open-sourced