Microscopic Spatial Intelligence: How a 7B Model Beat GPT-4 and Claude 3.5 on the MiSI-Bench Benchmark

Forum topic · ✨步子哥 · 2026-01-15

Summary

A new benchmark called MiSI-Bench (Microscopic Spatial Intelligence Benchmark) evaluates vision-language models' ability to perceive and reason about spatial relationships among microscopic entities such as molecules. In testing, Qwen2.5VL-7B-SFT, an open-source model with only 7 billion parameters fine-tuned on domain data, achieved the top average score of 62.96%, far surpassing GPT-4 (29.20%) and Claude 3.5 Sonnet (31.23%), though still below the human level of 81.18%. The model nearly matched perfection on basic spatial transformation tasks (99.84% on translation, 99.71% on rotation) and exceeded human performance on the challenging rotation-rotation task (~90% vs. human ~26%). However, all AI models performed poorly on tasks requiring domain knowledge, such as hydrogen bond identification and pocket-ligand interactions, where human experts dominated. The findings reveal that AI possesses strong geometric reasoning potential but lacks true understanding of chemical concepts, suggesting that integrating explicit scientific knowledge is essential for progress toward Scientific AGI, and demonstrating that targeted fine-tuning can let small models outperform giant general-purpose models in specialized domains.

Overview

MiSI-Bench (Microscopic Spatial Intelligence Benchmark) is a new evaluation framework designed to measure how well vision-language models (VLMs) can understand and reason about spatial relationships among microscopic entities such as molecules — an ability defined as Microscopic Spatial Intelligence (MiSI), described as a foundation for scientific discovery in structural biology, medicinal chemistry, and materials science.

Key Points

Benchmark design: MiSI-Bench contains 9 tasks built from ~4,000 molecular structures, generating 587,000 images and over 163,000 QA pairs. Tasks fall into three categories:
*Unit tasks*: translation, rotation, zooming
*Composite tasks*: translation-rotation, rotation-rotation (Rot-Rot)
*Scientific relational tasks*: residue-ligand interaction (positive/negative), interaction location, pocket-ligand interaction
Surprising winner: Qwen2.5VL-7B-SFT, a fine-tuned open-source 7B-parameter model, ranked first with an average score of 62.96% — the only model above 50%. Selected comparison results:

| Model | Avg. Score | | :--- | :--- | | Qwen2.5VL-7B-SFT | 62.96 | | Claude Sonnet 4.5 | 34.37 | | O3 | 33.65 | | Claude Opus 4 | 33.13 | | Claude Sonnet 3.5 | 31.23 | | GPT-4 | 29.20 | | Gemini-2.5-flash-lite | 16.00 | | Human level | 81.18 |

Superhuman geometric ability: The 7B model scored 99.84% on translation, 99.71% on rotation, 88.44% on translation-rotation, and 89.59% on rotation-rotation. Humans averaged only ~26% on Rot-Rot — near random guessing — while scoring 70.18% on rotation.
Failure on chemistry: On tasks requiring domain knowledge, all models fell far behind humans. Qwen2.5VL-7B-SFT scored 63.46% on positive residue-ligand interaction (human: 100%) and only 10.72% on pocket-ligand interaction (human: 82.78%). Performance on hydrogen bond identification was described as poor across all models.

What This Reveals About AI Cognition

1. "Giant in geometry, dwarf in chemistry": Models excel at pure geometric transformations but fail when reasoning requires chemical principles such as hydrogen bond directionality, donor-acceptor relationships, and binding pocket shape.

2. Pattern-matching limitations: Models tend to rely on spatial proximity as a proxy for molecular interactions, rather than understanding underlying chemical constraints — e.g., misjudging close atoms as strongly interacting without checking bond angles.

3. Domain adaptation, not spatial reasoning, is the bottleneck: The authors argue advanced VLMs lack visual priors for specialized structures like proteins, which blocks transfer of their general spatial abilities. Targeted fine-tuning removed this barrier for Qwen2.5VL-7B-SFT.

Implications

Small models can win in vertical domains: With high-quality domain data and supervised fine-tuning, a 7B open-source model decisively beat frontier closed models, offering a cost-effective path for resource-limited teams.
Explicit scientific knowledge is necessary for Scientific AGI: The results emphasize that scaling alone cannot produce scientific understanding; integrating structured knowledge (via knowledge graphs, graph neural networks, neuro-symbolic systems, or knowledge-aware training tasks) is a key future direction.
Benchmark value: MiSI-Bench fills a gap in evaluating microscopic spatial cognition, providing a reproducible standard for AI-for-Science research and highlighting the need for deep AI–chemistry–biology interdisciplinary collaboration.