Progressive Cramming: When One Embedding Stores 1,500 Tokens, Is the Model "Understanding" or Taking a "Shortcut"?
The Setup: A Counterintuitive Finding
Imagine compressing an entire Wikipedia article — 1,500 tokens — into a single embedding vector, then feeding that vector to a frozen LLM. Something remarkable happens: the model reproduces the article verbatim with less than 1% error.
This was demonstrated in 2025 by Kuratov et al. in the "Token Cramming" paper: a 1500x compression ratio, far beyond the ~10x lossless limit of traditional encoders. It seemed as if the Transformer's embedding space hid an untapped "information jungle."
But researchers at FusionBrain Lab asked an unsettling question: does the model actually "understand" the compressed information, or has it merely found a shortcut that makes reconstruction look like understanding?
Their paper, Progressive Cramming (arXiv: 2607.21231), offers a sobering answer: cramming's high-fidelity reconstruction is likely not "semantic encoding" but "fragile attention hijacking."
Fixing the Experimental Tool First: Progressive Compression
The original cramming methodology had a problem. It used a fixed token budget and a 99% reconstruction accuracy threshold. 99% sounds high — but how severe is the remaining 1% of errors?
The researchers uncovered a hidden disaster: that 1% of errors concentrates almost entirely at the earliest 0-1 token positions. Under teacher forcing, this looks harmless — after all, 99% of positions are correct. But once you switch to autoregressive generation (no teacher forcing; the model generates each next token itself), catastrophe strikes.
First token wrong → all subsequent tokens are generated from a wrong prefix → the entire sequence rapidly collapses. The result: even with 95-99% average convergence during training, greedy-decoding convergence at inference time is nearly 0%.
It's like copying answers on an exam: with a teacher pointing at the right answer (teacher forcing), you copy quickly and accurately. Once the teacher leaves (autoregressive generation), one wrong character at the start ruins everything.
To fix this, the researchers propose Progressive Cramming:
1. Start with 1 token; optimize the embedding until reconstruction is 100% perfect 2. Expand to 2 tokens, warm-starting from the previous embedding 3. Keep expanding one token at a time, building only on successful steps 4. Stop when perfect reconstruction becomes impossible
The benefit: a clear success/failure boundary. No more vague "99% correct" — either 100% reconstruction succeeds, or it fails at an identifiable step.
Finding 1: The Optimization Trajectory Is Low-Dimensional
First finding: the progressive-cramming optimization trajectory occupies a low-dimensional manifold in embedding space.
Embedding spaces are 2048-4096 dimensional (e.g., Llama-3.1-8B). Yet 99% of the trajectory's variance can be explained by just 30-100 PCA components.
What does this mean? Compressed embeddings don't wander all over high-dimensional space — they follow a narrow "rail." This rail is a structured subspace of the model's computational geometry; cramming succeeds because it finds this rail.
Finding 2: Compressed Embeddings Become "Attention Black Holes"
The more critical finding comes from attention-pattern analysis.
When compressed embeddings are fed to the model, the researchers observed that compressed embeddings capture 20-60% of attention mass in intermediate layers. They become massive "attention sinks," pulling in attention from surrounding tokens.
That sounds like a good thing — the compressed embedding matters, so the model attends to it. But causal analysis showed: this attention concentration is not the model "using" the compressed information; the compressed embedding is "hijacking" the model's attention.
Using "attention knockout" experiments — artificially masking the compressed embedding's attention connections at certain layers — the results were:
- Masking early layers: downstream capability recovers
- Masking late layers: almost no effect
Finding 3: Reconstruction Success ≠ Capability Preservation
If cramming were true "semantic encoding" — information stored in a way the model can read — a compressed prefix should preserve downstream capabilities. After all, the information is in there.
The researchers tested two multiple-choice tasks: HellaSwag and ARC-Easy. Results:
Under likelihood evaluation (the model compares option likelihoods): even with the original prefix still in context, prepending a compressed embedding causes a consistent but moderate accuracy drop.
Under generative evaluation (the model generates answers itself): capability collapses almost completely.
This contrast is crucial. If compressed embeddings truly encoded semantic information, generative mode should also exploit it. The generative collapse shows: the model isn't "reading" compressed information — it is being "guided" by it in some fragile way. Once teacher forcing's protection is gone, the guidance shatters.
Finding 4: Compression Capacity Grows with Depth and Width
Finally: cramming capacity is not unlimited — it is bounded by the capacity of the frozen reconstructor.
The researchers truncated pretrained models to their first N layers (with a brief fine-tune to repair the truncation point) and measured cramming capacity. Result: compression capacity grows monotonically with retained depth and model width.
This means cramming relies on the model's deep computational capacity — the more layers you give it, the more tokens it can "pretend to understand." This is itself counterevidence: if cramming were pure semantic encoding, it shouldn't depend so heavily on reconstructor depth — semantic information should be readable within the first few layers.
Rethinking Cramming: Not Encoding, But Manipulation
Assembled together, the paper draws a sobering conclusion:
> High-fidelity reconstruction reflects not dense semantic encoding, but the ease of "manipulating" the model through its attention mechanisms.
An analogy: cramming doesn't write information into the model's "memory book" for it to read; it finds a switch that directly triggers the model's "conditioned reflexes." Reconstruction looks perfect, but the model isn't "understanding" — it is being hijacked by a carefully optimized embedding vector that steers it along a specific trajectory to emit the correct token sequence.
Why does this distinction matter? Because it bears directly on whether compressed embeddings can be used in practice. If cramming were semantic encoding, you could compress long contexts into a few embeddings and the model would still use them. But if it's attention manipulation, any usage deviating from the training distribution causes collapse — exactly what the generative evaluation showed.
Methodological Contribution: Stop Using the 99% Threshold
Beyond its scientific findings, the paper makes an important methodological contribution: it exposes the blind spot of the "99% reconstruction accuracy" standard.
Under the fixed-budget + 99%-threshold protocol, you see "almost perfect." What you don't see: the 1% of errors sits at the most fatal positions (the earliest tokens), and under autoregressive generation those errors cause total collapse.
Progressive compression's value: by demanding 100% reconstruction and expanding token by token, it exposes that fatal 1% and gives a definitive answer to "does compression actually work?"
An Honest Assessment
The paper has limitations.
First, its conclusions rest mainly on a few models like Llama-3.1-8B. Although the authors claim consistency "across model families," whether other architectures (e.g., state-space models like Mamba) exhibit the same "attention hijacking" mechanism remains open.
Second, the "fragile attention manipulation vs. semantic encoding" dichotomy may be too simple. Reality may be a mixture — compressed embeddings may contain some semantic information while also exploiting attention shortcuts. The causal analysis is strong but doesn't fully rule out "partial semantic encoding."
Finally, progressive compression, while fixing the 99% threshold's blind spot, has its own limitations: the token-by-token expansion is computationally expensive, and the warm-start strategy may introduce its own biases (each step's solution depends on the previous step's local optimum).
These are minor issues. The paper's real contribution: it pulls a seemingly magical phenomenon (1500x compression) back to earth, revealing the imperfect truth possibly hidden behind "perfect reconstruction".
A Deeper Takeaway
The paper raises a more general question: in AI research, the gap between "looks like it's working" and "actually working" may be far larger than we assume.
Cramming's 99% reconstruction rate, LLM benchmark scores, RLHF models becoming more "polite" — how many of these "looks-like-it-works" signals are genuine semantic understanding, and how many are some form of "attention hijacking" or "reward shortcut"?
Progressive Cramming offers a method: when you see an "almost perfect" result, ask where the remaining 1% is and how fatal it is. The answer may make you rethink what that "perfection" really means.
---
Paper: https://arxiv.org/abs/2607.21231 Code: https://github.com/FusionBrainLab/progressive_cramming HTML version: https://arxiv.org/html/2607.21231v1 Authors: FusionBrain Lab