Embedding Models Measure in Peculiar Ways: Physical Quantities Are Weakly Modeled in Text Embedding Spaces

Forum topic · 小凯 · 2026-09-19

Summary

This arXiv paper (2609.20821) by Juri Opitz and Andrianos Michail investigates whether text embedding spaces capture physical measurements—mass, distance, time, and volume—which possess unique, objective notions of semantic equivalence and distance. The authors find that physical measurement is only weakly modeled in embedding spaces, and that instead quite peculiar measurement patterns emerge. Further analysis indicates that embedding representations of physical quantities are strongly influenced by superficial string similarity rather than true semantic magnitude, and recalibrating similarity does not substantially improve alignment with physical scales. The work has implications for retrieval, reasoning over quantities, and the evaluation of embedding quality, suggesting caution when using embeddings for tasks requiring precise numeric or physical understanding.

Embedding Models Measure in Peculiar Ways

Research area: NLP Authors: Juri Opitz, Andrianos Michail Published: 2026-09-17 arXiv: 2609.20821

Abstract

Embedding spaces define notions of semantic similarity and distance. We study whether those embeddings reflect physical measurements of mass, distance, time and volume, which admit a unique, objective notion of semantic equivalence and distance. We find that physical measurement is only weakly modeled in the embedding space, and that instead quite peculiar measurement patterns can be observed. Further analysis indicates that embedding representations of physical measurements are strongly influenced by superficial string similarity, and recalibration of similarity does not substantially improve the alignment.

Key Findings

Physical quantities (mass, distance, time, volume) are only weakly represented in embedding spaces, despite having objective semantic equivalence and distance relations.
Embedding spaces exhibit peculiar measurement patterns instead of faithful physical scaling.
Representations are strongly driven by superficial string similarity rather than the underlying semantic magnitude.
Recalibrating similarity does not substantially improve alignment with physical measures.

Significance

The results suggest that practitioners should be cautious when relying on embeddings for tasks involving numeric reasoning or physical quantities (e.g., unit conversion, quantity comparison, retrieval over measurement-bearing text), since embedding geometry does not reliably encode objective physical scale.

*Auto-collected on 2026-09-19.*