Meaning Isn't in the Vector, It's in the Computation: A Direct Challenge to RAG's Geometric Assumption
An Awkward Fact
You build a RAG (retrieval-augmented generation) system with a good embedding model, encode your document corpus into vectors, and store them in a vector database. A user asks "how do I take a screenshot on an iPhone," and you retrieve a document titled "A detailed guide to iPhone screenshots" — cosine similarity 0.92, a perfect hit.
But the same corpus also contains: "How to capture the screen while taking photos on an iPhone." Its meaning is nearly identical to the user's question, yet its cosine similarity is 0.71, ranking 15th — truncated away.
You might think: just switch to a better embedding model.
This paper by Jiaqi Deng tells you: it doesn't matter which model you switch to. The problem isn't the model — it's the entire paradigm. "Meaning identity" has never lived in a single sentence's vector.
Two Worldviews
The paper splits the problem cleanly. For the question "do two sentences express the same meaning," there are two views:
The geometric worldview: every sentence has a vector; "same meaning" is a geometric property of two vectors, computable via cosine similarity. The entire RAG, dense retrieval, and duplicate detection industry is built on this assumption.
The computational worldview: meaning identity is not a property of individual sentences, but a relation the network computes when two sentences appear together in the same forward pass.
This isn't a rhetorical dispute. The paper designs two decisive discriminating experiments:
- Late fusion: encode both sentences separately, concatenate the two vectors, and feed them to a linear probe. If meaning really lives in the vectors, a linear probe should be able to read it out.
- Shuffle control: replace sentence B with an unrelated sentence while keeping the format unchanged, and run the joint forward pass. If the probe is reading template format rather than the pairwise relation, it should still work after shuffling.
Let the Numbers Speak
The paper's core results table is striking. On PAWS-X (a dataset specifically built to test paraphrase confounds):
| Model | Cosine | Linear probe (late fusion) | Joint forward | Shuffle | |------|------|----------------------|---------|---------| | Qwen2.5-14B | 0.614 | 0.615 | 0.950 | 0.496 | | Qwen2.5-7B-Inst | 0.571 | 0.612 | 0.961 | 0.526 | | Llama-3-8B | 0.530 | 0.514 | 0.897 | 0.542 | | GPT-2 XL | 0.521 | 0.513 | 0.756 | 0.457 |
Key findings:
1. Joint forward passes crush everything. Concatenate two sentences, run one forward pass, and read "same meaning or not" from the last token's hidden state: AUC jumps from ~0.55 to ~0.95. No fine-tuning, no training — just a linear probe.
2. Shuffle is the fatal blow. Swapping sentence B for an unrelated one while keeping the format drops AUC to 0.50 immediately. The model reads the genuine pairwise relation, not the template.
3. 1.5B is enough. Qwen2.5-1.5B's joint-probe AUC is 0.947 — nearly identical to 32B's 0.938. This capability saturates around 3B parameters.
4. GPT-2 XL already has it. Only 0.76, but far above cosine's 0.52. This computational ability isn't exclusive to large models — it's a basic capability of language models.
Why This Matters
Think about your RAG system. A corpus of 1M documents, a user query, embedding-encoded into a vector, cosine similarity search in the vector store, top-k returned.
The implicit assumption: same meaning = high cosine similarity.
But the paper shows: for "reworded but same meaning" hard cases, cosine similarity is essentially random guessing. Retrieval-specialized models — BGE, E5, GTE, MiniLM — achieve only 0.55–0.65 AUC on PAWS-X.
You might say: then use a reranker. The paper tested that too. BGE-reranker-large does reach 0.94, but MS-MARCO and Jina rerankers remain in the 0.55–0.64 range. And a reranker is, fundamentally, a joint forward pass — a cross-encoder over the concatenated query and document. Rerankers work precisely because they inadvertently adopt the "computational worldview."
A Deeper Finding: Cross-Model Consistency
The paper includes a spine-chilling experiment. Models from different families, scales, and independent training runs (Llama, Mistral, Qwen, Phi, GPT-Neo, OPT) — all run as joint forward passes — produce highly consistent judgments: not just matching labels, but matching errors.
What does this mean? Language models trained by different teams, on different data, with different architectures, have converged on the same computation for "is this pair synonymous?" This is not an accidental engineering artifact — it's something deeper.
The paper also ran a distillation experiment: using a 1.5B student model with unlabeled sentence pairs scored by a teacher, the student recovers the same judgment capability. But no linear combination of the teacher's individual sentence vectors achieves this.
The relation is distillable; the vectors are not.
Implications for Engineering Practice
This paper isn't purely theoretical — it has direct implications for RAG system design:
1. The bi-encoder paradigm has a ceiling. Encoding documents into vectors upfront and searching by vectors at query time has a hard flaw on "reworded but same meaning" queries. Fine-tuning embedding models helps, but the paper shows fine-tuned models improve on PAWS while getting worse on QQP — you're just fitting a particular dataset.
2. Rerankers aren't a nice-to-have; they're a necessity. But rerankers are slow and expensive — you can't run a cross-encoder over 1M documents. So real systems are two-stage: bi-encoder coarse filtering to top-100, then reranker fine ranking. That paradigm is correct, but you need to know the coarse stage may already have discarded the truly synonymous documents.
3. Meaning is an operator, not a vector. If you're building synonym detection, don't try to read it out of a single vector. Either run a joint forward pass or train a dedicated pair reader.
An Honest Assessment
The paper has limitations:
1. Only PAWS-X was tested. PAWS is specifically designed for "high lexical overlap, different meaning" — the hardest case for embeddings. In more common semantic search scenarios (e.g., "iPhone screenshot how-to" vs. "how to screenshot on an iPhone"), embedding models may already be good enough.
2. The engineering cost of joint forward passes. The paper proves joint forward passes can read out meaning, but for RAG this means running the model over the query concatenated with every candidate document — extremely expensive.
3. No new RAG architecture proposed. The paper identifies the problem but offers no replacement. Rerankers are the existing approximate solution, but the paper hints that better pair reader architectures may exist.
Even with these limitations, the paper's value is clear: with a clean experimental design, it challenges an assumption an entire industry relies on. Next time you tune your embedding model's cosine similarity threshold, ask yourself: are you measuring "semantic similarity," or just "phrasing neighborhoods"?
---
Paper: Computation Over Geometry: Meaning Identity Is Computed, Not Shipped in the Embeddings
Author: Jiaqi Deng (independent researcher)
Core conclusion: Meaning identity is not a geometric property of individual sentence vectors, but a relation computed when two sentences share a forward pass. The "geometric assumption" underpinning the entire RAG industry is wrong in the hard cases.