OmniScientist Deep Dive: When AI Gains Senses and Becomes a Real Scientist
Prologue: The Blindfolded Sage
Imagine a scientist widely hailed as a "genius" who has never looked through a microscope at dividing cells, never heard the rumble of seismic waves, never touched a material's crystal structure. All their research depends on notes handed to them—filled with numbers and textual descriptions.
Could they make great discoveries? Perhaps, relying on powerful reasoning. But they could never directly feel the texture of data, catch an anomalous tremor in an overlooked waveform, or notice patterns in microscope imagery that others missed.
This is the predicament of most current "AI scientists." They can read papers, write code, generate hypotheses, even produce respectable research reports. But like the blindfolded sage, they sit behind a thick curtain separating them from the raw evidence of the real world. They handle preprocessed numbers, summaries, and labels—never truly "seeing" a nebula's structure in an astronomical image, "hearing" an abnormal rhythm in an EEG trace, or "feeling" the tension between atoms in a molecular dynamics simulation.
This is exactly the barrier OmniScientist aims to break.
Act I: We Need Observers, Not Faster Test-Takers
The Gap Between Test-Taker and Observer
Current systems, such as heavily hyped "AI scientist" tools, are essentially sophisticated "test-takers":
- Sakana AI's AI Scientist: good at generating hypotheses and code, but requires humans to convert raw data into text descriptions or numeric tables first.
- GPT-4 with tool chains: can write code and search literature, but when understanding a complex medical image, it relies on text labels from image-recognition models rather than directly "gazing" at the image.
- Automated experiment systems: can control robots, but analyze sensor-output numbers rather than the full phenomenon.
- Images: microscopy, astronomy, SEM images, medical imaging
- Signals: EEG, seismic waves, spectra, any 1D time series
- Audio: animal calls, mechanical faults, acoustic features
- Video: dynamic processes from cell division to fluid dynamics
- 3D structures: molecules, protein folding, crystal lattices, point clouds
- Trajectories: particle motion, animal migration, celestial orbits
- Tables: experimental data, statistics, multivariate datasets
- Formulas: equations, physical laws, chemical formulas
- Graph networks: social networks, molecular interaction networks, knowledge graphs
- Li, B., Fei, H., Ju, T., Lee, M. L., & Hsu, W. (2026). OmniScientist: An Omni-Modal Omni-Discipline AI Scientist. arXiv preprint arXiv:2608.13558.
- Sakana AI. (2024). The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv:2408.06292.
- Feynman, R. P. (1965). The Character of Physical Law. MIT Press.
- Krenn, M., et al. (2022). Scientific discovery in the age of artificial intelligence. Nature, 620(7972), 47-60.
- Wang, H., et al. (2024). Large Language Models for Scientific Discovery: A Survey. arXiv:2402.10091.
Their core limitation: they are blind at the critical stages of the research lifecycle.
What Does Scientific Discovery Actually Require?
Feynman famously described watching a plate tossed in the air at a Cornell cafeteria and, out of boredom, computing the relation between its wobble and spin—a calculation that became the mathematical foundation of his Nobel-winning quantum electrodynamics work. The lesson: real discovery often begins with direct perception of phenomena, not processing secondhand information.
A scientist's daily lab life involves observing samples directly, listening for abnormal equipment sounds, feeling material textures, integrating multiple sensory channels, and tracking how phenomena change over time. Current AI scientists possess almost none of these abilities.
Act II: OmniScientist's Five Senses — The Omni-Modal Perception System
OmniScientist's core innovation: giving AI genuine "senses." Not shallow cat-in-picture vision, but the ability to examine an astronomical image and notice "a faint ring structure in the upper right that didn't appear in previous observations." Not mere audio transcription, but capturing "an anomalous frequency component 3.2 seconds after the P-wave arrives."
The Perception Layer is designed to interface with raw evidence directly, not preprocessed features. It handles:
Why Raw Evidence Matters
Consider analyzing a novel superconductor's electron microscopy images. The traditional pipeline has humans extract features (lattice constants, defect densities, phase boundaries) and feed numbers to the AI. But human feature extraction embeds prior judgments—what if the crucial information lies in the ignored regions, or an unprecedented microstructure gets filtered out by known taxonomies?
OmniScientist instead receives the raw pixel data and autonomously discovers patterns, anomalies, and correlations across the full image space. The experimental results support this strongly: the version that directly perceives raw evidence improved on all 7 evaluation dimensions versus the "blind" version limited to preprocessed features, winning 85% of head-to-head comparisons.
Act III: Three Agents in Concert — Automating the Research Lifecycle
OmniScientist coordinates three specialized agents sharing one perception layer in a deterministic pipeline:
Ideation Agent
Proposes valuable research questions based on raw observations, balancing novelty, feasibility, importance, and precision. Each idea passes an automated Idea Check: the idea is translated into code and executed to verify its basic logic—like an automated thought experiment.Experiment Agent
Designs experiments, runs statistical analyses, validates reliability, and ensures traceability. A key innovation is the Rigour Check, which prevents p-hacking by enforcing statistical norms at the code level: automatic multiple-comparison correction, pre-registered analysis plans, recording all intermediate steps for numerical traceability, and verifying execution provenance for reproducibility.Writeup Agent
Builds a clear, persuasive, standards-compliant argument chain: translating findings into peer-readable narrative, citing related work, honestly reporting limitations (part of Claim Check), and generating high-quality figures. Claim Check ensures every major claim is backed by data or derivation, curbing overselling.Act IV: From Lab to Real World — 36 Case Studies
The evaluation was intentionally harsh: 36 real-data cases across 5 disciplines and 4 evidence types.
Disciplines: biomedical (gene expression, medical imaging, protein structure), physical sciences (astronomy, materials, quantum computing), Earth and environmental sciences (climate, seismic signals, remote sensing), engineering (robot sensors, network traffic, chip verification), and social sciences (economic data, social networks, behavioral experiments).
Evidence types: spatial (images, 3D, distributions), temporal (signals, trajectories, time series), relational (graphs, matrices, tables), and procedural (code, algorithms, computation).
Across all 36 cases, OmniScientist completed the full pipeline from raw data to complete paper, with an average paper score of 6.3/10—in academic review terms, roughly "solid work, minor revisions." For a fully automated system, this is a remarkable achievement: AI-generated papers reaching the threshold quality of conference or journal submissions.
The ablation results are even more striking: direct perception of raw evidence beat preprocessed features on all 7 dimensions, winning 85% of head-to-head comparisons. Letting AI "see" raw evidence is not a nice-to-have—it is a qualitative leap.
Act V: Why This Matters — A Potential Paradigm Shift
From Human-Assisted AI to AI-Assisted Science
Historically AI served as a tool while humans remained central to research. OmniScientist demonstrates AI as a direct executor of research, from observation to paper. The human role may shift toward setting direction, evaluating AI findings, and making key judgments. This is not about replacing scientists but massively amplifying their capacity—e.g., continuously monitoring ten automated microscopes' output and flagging interesting findings for human investigation.
A Cross-Disciplinary Accelerator
Human scientists specialize narrowly; OmniScientist can simultaneously "observe" protein structures and nebula images, surfacing hidden connections across seemingly unrelated phenomena—much as Malthusian economics inspired Darwin, and Schrödinger's *What Is Life?* inspired molecular biology.Democratizing Evidence Quality
Reliable automated analysis of complex raw data could lower barriers to high-quality research, letting resource-limited researchers participate in frontier discovery—though it also raises challenges: ensuring reliability and preventing an influx of "AI-generated junk science."Epilogue: The Road Ahead
The average 6.3 score also reveals room for improvement. True breakthrough, Nobel-level discoveries require deep intuitive understanding of a field, taste for genuinely important problems, courage to speculate with insufficient evidence, and resilience in failure—qualities current AI lacks. But OmniScientist proves the first step—letting AI directly perceive the world—has been taken.
As Feynman said, "What I cannot create, I do not understand." By working from direct observation to autonomous experimentation, OmniScientist embodies that philosophy. Future scientists may be human–machine teams: human creativity and intuition, plus AI's tireless observation and precise execution.