MiSI-Bench: How a 7B Model Beat GPT-4 on Molecular Spatial Intelligence, and What It Reveals About AI's Blind Spots

Forum topic · ✨步子哥 · 2026-01-15

Summary

A new benchmark called MiSI-Bench, dubbed a 'microscopic gaokao' for AI, tests vision-language models' ability to perceive and reason about spatial relationships among microscopic entities like molecules. Surprisingly, Qwen2.5VL-7B-SFT, an open-source model with only 7 billion parameters fine-tuned on the benchmark, ranked first with an average score of 62.96%, far surpassing GPT-4 (29.20%), Claude 3.5 Sonnet (31.23%), Claude Sonnet 4.5 (34.37%), and O3 (33.65%). The model nearly aced basic geometric tasks (99.84% on translation, 99.71% on rotation) and even beat humans on the composite rotation-rotation task (89.59% vs. ~26%, near random guessing). However, all AI models failed on tasks requiring domain knowledge, such as hydrogen bond identification: Qwen2.5VL-7B-SFT scored only 10.72% on pocket-ligand interaction versus 82.78% for human experts, while humans scored 100% on positive residue-ligand interaction. The benchmark, built from ~4,000 molecular structures with 587,000 images and 163,000 QA pairs across nine tasks, exposes a stark contrast: AI shows geometric talent but chemical incompetence, revealing that current models rely on pattern matching and spatial proximity rather than understanding scientific principles. The findings argue that integrating explicit domain knowledge is essential for progress toward Scientific AGI.

Key points

What is MiSI-Bench?

MiSI-Bench (Microscopic Spatial Intelligence Benchmark) evaluates vision-language models' (VLMs) ability to perceive and reason about spatial relationships among invisible microscopic entities like molecules — defined as a foundational capability for scientific discovery. The dataset is built from ~4,000 molecular structures, generating 587,000 images and over 163,000 question-answer pairs across nine tasks:

Unit tasks: translation, rotation, zooming
Composite tasks: translation-rotation, rotation-rotation
Scientific relational tasks: positive/negative residue-ligand interaction, interaction location, pocket-ligand interaction

Headline results

| Rank | Model | Avg. Score | | :--- | :--- | :--- | | 1 | Qwen2.5VL-7B-SFT | 62.96 | | 2 | Claude Sonnet 4.5 | 34.37 | | 3 | O3 | 33.65 | | 4 | Claude Opus 4 | 33.13 | | 5 | Claude Sonnet 3.5 | 31.23 | | 7 | GPT-4 | 29.20 | | 11 | Gemini-2.5-flash-lite | 16.00 | | – | Human level | 81.18 |

A fine-tuned open-source 7B model (Qwen2.5VL-7B-SFT) was the only model above 50%, more than doubling GPT-4's score.
Nearly perfect on basic spatial tasks: 99.84% (translation), 99.71% (rotation); 88.44% (trans-rot), 89.59% (rot-rot).
On the rotation-rotation task, it hit ~90% while humans scored ~26% — near random guessing.

Geometric talent vs. chemical incompetence

Geometry: Fine-tuned models excel at inferring 3D transformations from 2D images, even surpassing human experts on purely geometric tasks. The study suggests frontier VLMs' weakness stems from a *domain adaptation* barrier (lack of visual priors about protein structures) rather than absent spatial reasoning ability.
Chemistry: All models performed poorly on tasks requiring domain knowledge. Qwen2.5VL-7B-SFT scored 63.46% on positive residue-ligand interaction (humans: 100%) and just 10.72% on pocket-ligand interaction (humans: 82.78%). Hydrogen bond identification was a particular failure mode.
Models rely on spatial proximity rather than chemical principles — they performed relatively better on negative interaction samples simply because residues and ligands were far apart, and cannot apply constraints like hydrogen-bond directionality and donor-acceptor geometry.

Implications

1. Small models can win in vertical domains: Targeted supervised fine-tuning on high-quality domain data lets a 7B model outperform frontier models with hundreds of billions of parameters — challenging "scale worship" and lowering barriers for resource-limited research groups. 2. Benchmarks matter: MiSI-Bench fills a gap in evaluating AI's cognitive abilities in the microscopic world, providing a reproducible standard for the AI-for-Science community. 3. Pattern matching is not understanding: Current AI is data-driven rather than theory-driven; it finds correlations but cannot build causal models grounded in physics and chemistry. Its strong macroscopic-world performance does not transfer automatically to the microscopic domain. 4. Integrating explicit domain knowledge is essential for Scientific AGI. Open challenges include representing abstract concepts (chemical bonds, molecular force fields) in machine-usable form, with future directions spanning neuro-symbolic architectures, knowledge-guided pretraining tasks, large scientific multimodal datasets, and human-AI collaborative learning.

Bottom line

AI today can serve as a scientist's "eyes" — inferring 3D structure from 2D images with superhuman geometric precision — but not yet as a scientist's "brain". Bridging the gap between geometric pattern recognition and genuine chemical understanding is the central challenge for AI-driven scientific discovery.