If you ask an LLM to produce text embeddings—turning a passage of text into a vector—you'll find it does surprisingly poorly. How can a model that writes poetry, reasons, and codes fail at such a "basic operation" as converting text into vectors?
A research team from Renmin University and Lenovo found the answer, and it's hidden inside the model itself.
A Counterintuitive Finding
The researchers observed a strange phenomenon: when you project LLM-generated text embeddings back into the vocabulary space, these embeddings align strongly with high-frequency, low-information tokens.
What does this mean? Imagine asking the model to embed the sentence "Quantum computing will be commercially viable by 2030." You'd expect the vector to capture semantics like "quantum computing" and "commercialization." Instead, the vector is closer to the representations of high-frequency words like "the," "is," or "was."
This isn't a bug—it's the LLM's "factory settings" at work.
Reverse-Engineering the "Average Token"
The researchers' approach was clever: since embeddings correlate with word frequency, could we reverse-engineer an "average token"—the hidden state of the most typical token in the training corpus?
Using the LLM's unembedding matrix (the final layer that maps hidden states back to the vocabulary), combined with the word-frequency distribution of the training corpus, they successfully reverse-engineered this "average token."
Then they did something even more elegant—Logit Spectroscopy on this "average token." Like a prism splitting white light into a spectrum, they decomposed the singular values of the unembedding matrix into different frequency bands.
The result: the edge spectrum subspace—dimensions corresponding to the largest and smallest singular values—is the culprit driving the emergence of high-frequency tokens.
EmbedFilter: A Suspiciously Simple Solution
Once the cause was found, the fix was straightforward. The researchers proposed EmbedFilter: a linear transformation that filters out the edge spectrum subspace, keeping only the middle "body spectrum."
Concretely, after SVD decomposition of the unembedding matrix, remove the dimensions corresponding to the largest and smallest singular values, build a projection matrix from the remaining dimensions, and project the original embeddings onto it.
That's it. No fine-tuning, no extra training, no changes to the model's parameters. One linear transformation, done.
How Well Does It Work?
Results on MTEB (Massive Text Embedding Benchmark):
- Qwen-2.5 + EmbedFilter: average score improved from 52.23 to 55.39 (+6.1%)
- Llama-3.1 + EmbedFilter: average score improved from 56.73 to 58.79 (+3.6%)
Even better, because EmbedFilter effectively reduces dimensionality (some dimensions are removed), storage and retrieval speed naturally improve while embedding quality gets better. It's like denoising a photo: removing noise makes the file smaller and the image sharper.
Why This Finding Matters
This paper reveals a deep mechanistic insight: an LLM's unembedding matrix is not merely an "output layer"—it encodes a latent space that actively "writes" high-frequency tokens into the embedding space.
In other words, LLMs are bad at embeddings not because of insufficient capability, but because their architecture has a built-in bias that naturally pulls embeddings toward high-frequency words. This bias is useful for language generation ("the" is indeed more common than "quantum"), but harmful for semantic representation.
The elegance of EmbedFilter is that it doesn't fight this bias—it filters it out after embedding generation. The model itself is completely unchanged; you just remove the "noise" at the final step.
This reminds me of an analogy: your ears are naturally more sensitive to certain frequencies (like a baby's cry), which makes evolutionary sense, but when listening to music you'd want an equalizer to tone them down. EmbedFilter is the equalizer for LLM embeddings.
---
Paper: Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings (arXiv: 2606.07502) Link: https://arxiv.org/abs/2606.07502 Code: https://github.com/CentreChen/EmbFilter Authors: Songhao Wu, Zhongxin Chen, Yuxuan Liu, Heng Cui, Cong Li, Rui Yan (Renmin University / Lenovo / Wuhan University)