The Two Banks of the Computation River: How LLMs Allocate Compute Internally
> If we imagine a language model's forward pass as a river, each token is a boat. Different stretches of the river have different speeds and depths—some turbulent, some calm, some so shallow a boat can almost skim across the surface. > > But we have never had a topographical map of this river. We know how long it is (number of layers) and how wide it is (number of attention heads), but not how much energy boats actually consume at each stretch. > > A new paper uses a method called s-Trace to draw, for the first time, a computational density map of this river.
---
| Item | Detail | |------|--------| | Paper | Tracing Computation Density in LLMs | | Authors | Corentin Kervadec, Iuliia Lysova, Iuri Macocco, Marco Baroni, Gemma Boleda | | Institutions | Universitat Pompeu Fabra (Barcelona), ICREA | | arXiv ID | 2605.27033 | | Submitted | May 26, 2026 | | Category | cs.CL | | Core findings | LLM computation splits into a Construction Phase (0.01%–1% of model) and a Refinement Phase (1%–100%); a minimal core of just 0.1% of the subgraph recovers the top-1 predicted token; attention is extremely sparse—99% of attention edges are redundant; computational density correlates positively with input uncertainty and negatively with token frequency |
---
1. Measuring the Depth of the River
Internally, a large language model can be described as a computation graph: each layer has n token nodes, connected by attention edges, MLP edges, and residual edges. A 32-layer model processing a 100-token input yields a graph with 6,302 nodes and over 5 million edges.
A forward pass traverses this entire graph—no shortcuts, no skipping—regardless of whether the input is a physics formula or a single period.
The paper asks: is full-graph traversal actually necessary? If the model is allowed to use only a subset of edges, how well can it reproduce the full output?
Their method, s-Trace, takes a computation budget s (the number of edges, as a relative fraction of the full graph) and answers: among all subsets of s edges, which group produces an output probability distribution closest to the full model's?
Framed directly: not "does the model break if we remove these edges?" but "can the model still be correct with only these edges?"
They ran this on 10 models (7B–14B parameters, spanning seven families: Qwen, Llama, DeepSeek, Phi, OLMo, Mistral, etc.), extracting traces at 26 granularity levels for 5,000 Wikitext inputs—about 1.3 million traces in total. Together, these traces draw the first LLM computational density curve.
---
2. Two Phases: Scaffolding and Fine Finishing
The first structural finding is a two-phase regime.
As trace size grows from tiny (0.01% of the graph, ~500 edges) to 1%, reconstruction error drops with a very steep slope. This interval is the Construction Phase: every added edge yields a significant jump in prediction quality—extremely high marginal utility.
Beyond the 1% threshold, the curve enters a plateau with only gradual improvement: the Refinement Phase, where marginal utility becomes weak and diffuse.
At the boundary lies a remarkably precise point—0.1% of the full graph, which the paper calls the Minimal Core. Here, the trace can already accurately predict the full model's highest-probability token. By a "correct token ranked first" criterion, a 7B model needs only 0.1% of its nodes for most inputs.
But decent AI-generated text requires more than the top-1 token—it needs distributional richness (nucleus sampling at top-60%). Reaching that level requires a trace of about 1%—exactly where the Construction Phase ends and the Refinement Phase begins.
Intuition: the Construction Phase is the circuit your brain uses to decide "the next word is probably X." The Refinement Phase is the circuit used to say "not just X, but possibly Y and Z, with subtle differences in style and precision."
---
3. What Lives Inside the Minimal Core
Dissecting the Minimal Core reveals a highly stable structure, consistent across inputs. For Qwen3-14B at s=0.1%, the top 20% most frequently used components (certain residual edges, MLP edges, or attention heads) account for 80% of all trace-edge traffic. The core is a small group of fixed "elder nodes" activated under any input.
By layer, the core is dominated by early layers: initial and early-middle layers account for 80% of edges. This doesn't mean late layers are useless—their contribution grows only in the Refinement Phase. During construction, the model leans on shallow, general feature extraction to land a rough prediction quickly.
The most striking finding concerns extreme sparsity of attention edges. In the full graph, attention edges exceed 99% of all edges—every token attends to every other at each layer. But in the Minimal Core, attention edges make up only about 50%, with MLP edges at 20% and residual edges at 30%—far more balanced than the full graph.
This implies a counterintuitive fact: the vast majority of attention computation is redundant. The attention transfer the model truly needs happens between only a few tokens; most token-to-token "handshakes" can be skipped entirely when forming the core prediction.
---
4. Which Inputs Demand More Compute
If different inputs need different compute, "full-graph traversal" is not the optimal strategy. The paper identifies two clear factors:
Uncertainty: Using Shannon entropy of the full model's output distribution, computational density (quantified by the AUC of the error-vs-trace-size curve) shows significant positive correlation with entropy across all 10 models—mean correlation 0.22. Low-entropy inputs (model is confident) need very few edges; high-entropy inputs (model hesitates among candidates) need larger trace budgets.
Token frequency: High-frequency tokens (common collocations like "the", "is") show consistent negative correlation with computational density across all model families—correlations in [-0.45, -0.22]. Common tokens require little compute; rare tokens (like "zymurgy") require the model to summon more internal resources.
A notable cross-model comparison: computational density distributions are highly consistent across models on the same inputs—cross-model correlations from 0.26 to 0.71, mean 0.52. This suggests density depends on properties of the input itself, not on a specific architecture. Where LLaMA struggles, Qwen likely struggles too. This consistency hints that computational density could serve as a universal measure of linguistic complexity.
---
5. What This Means for Understanding AI
The paper's deeper question is not "can inference be faster?"—though it does give efficiency optimization a precise direction—but: is an LLM's thinking really a dense, one-pass computation?
The answer is no. Its thinking goes coarse-to-fine, scaffolding first and details later, shallow statistics first and contextual rules deeper. And it wastes enormous compute where none is needed.
Together, these points reframe LLM computation from a black box into a two-phase, input-sensitive hierarchical system—structurally similar to human language processing, where high-frequency words are also processed faster and more cheaply (Gibson et al., 2019), and uncertainty recruits more cognitive resources.
The paper doesn't claim LLMs think like humans. But it suggests: efficient processing—"use more when needed, less when not"—is not a strategy monopolized by human brains. It may be a natural convergence point for any general information-processing system.
---
6. Honest Blind Spots
What's clear:
- s-Trace's methodology is sound: greedy backward edge search from output nodes, with L1 norms quantifying each edge's contribution; it outperforms random baselines in ablations.
- The two-phase structure holds consistently across all 10 models—not an artifact of one architecture.
- The 0.1% minimal core is impressive, and as a conservative reference line for "minimal faithful fidelity" it is reasonable.
- Cross-model density consistency (mean 0.52) meaningfully suggests input complexity drives compute demand.
- Is s-Trace truly finding the optimal subgraph? Greedy search plus L1 norms is a computationally feasible approximation, not a global optimum—some structurally different subgraph might do better at the same budget. Ablations cover random baselines only, not stronger search strategies (genetic algorithms, beam search).
- What does "0.1% recovers top-1" mean practically? Top-1 recovery only supports greedy decoding of the next token—not generation quality. Real generation with nucleus sampling (top-60%) requires ~1% traces. A core-only model would autocomplete accurately but miss distributional richness in creative writing.
- Why is attention so sparse? The paper shows the finding clearly (99% of graph vs. 50% of core) but offers no mechanistic explanation—is it semantically distant tokens with near-zero weights, or information the model never learned to pass?
- Non-English data: experiments used only English Wikitext. Whether Chinese, Arabic, or Japanese show different density curves may affect cross-lingual optimization strategies.
- Larger models: experiments cover 7B–14B. Whether the two-phase structure holds at 70B, 405B, or beyond—and whether the minimal core shrinks further—is a key open question for efficiency work.
What's unclear:
7. A New Channel
Throughout AI research history runs a quiet thread: we keep chasing bigger models and more compute, yet every so often a paper comes along saying—wait, you may only be using a tenth, or a hundredth, of this model.
This doesn't necessarily mean "big models are wasteful." Some redundancy during training is necessary—without the 99%, the model would never learn the 1% core. But redundancy at inference time should be avoidable: if so few edges suffice for core prediction, why force every token through all 5 million edges?
s-Trace doesn't answer "how to do it," but it answers "how far one can go." Its 1.3-million-trace efficiency curve provides a precise, quantifiable reference frame for next-generation sparse inference, conditional computation, and interpretability research.
And it quietly poses a bigger question: if an LLM has so many "unused" circuits, what are they doing? Backup redundancy (like an extra blood vessel against stroke), useless byproducts (an evolutionary tail), or purposes we can't yet see with crude metrics like top-1 accuracy?
The paper doesn't answer this. But it places the question where it can be measured. That alone is its most precious contribution.
---
| Item | Detail | |------|--------| | Core contributions | (1) s-Trace method, first large-scale quantification of LLM computational density; (2) two-phase computation: Construction (0.01%–1%) and Refinement (1%–100%); (3) Minimal Core (0.1% of graph) recovers top-1 token predictions; (4) extreme attention sparsity—99% of attention edges redundant; (5) computational density positively correlated with input uncertainty, negatively with token frequency | | Key limitations | Greedy search not globally optimal; English-only data; models limited to 7B–14B; 0.1% core supports only greedy decoding; no mechanistic explanation of attention sparsity; alternative search strategies unexplored |
References: 1. Kervadec, Lysova, Macocco, Baroni & Boleda, "Tracing Computation Density in LLMs", arXiv:2605.27033, 2026. 2. Veit, Wilber & Belongie, "Residual Networks Behave Like Ensembles of Relatively Shallow Networks", NeurIPS, 2016. 3. Lad, Lee, Gurnee & Tegmark, "The Remarkable Robustness of LLMs: Stages of Inference?", NeurIPS, 2025. 4. Gibson et al., "How Efficiency Shapes Human Language", Trends in Cognitive Science, 2019. 5. Ferrando & Voita, "Information Flow Routes: Automatically Interpreting Language Models at Scale", EMNLP, 2024.