OmniScientist: An Omni-Modal Omni-Discipline AI Scientist
Paper: OmniScientist: An Omni-Modal Omni-Discipline AI Scientist Authors: Bobo Li, Hao Fei, Tianjie Ju, Mong-Li Lee, Wynne Hsu arXiv: 2608.13558 (cs.AI, cs.CL)
The Problem: 'Blind' AI Scientists
Current AI scientist systems can automate increasingly complete research workflows—hypothesis generation, code execution, manuscript preparation—but they typically reason only over text, code, labels, or precomputed summaries. The scientifically decisive information—spatial relations, temporal dynamics, cross-channel correlations, and procedural signals in raw data—remains invisible to them. It is like a detective who can only read written reports but cannot examine photos, recordings, or 3D fingerprints.
Science itself shows why this matters: Watson and Crick's discovery of the DNA double helix came from directly *seeing* X-ray diffraction patterns (Photo 51), not from reading textual summaries. Scientific discovery depends on direct perception of raw evidence, not second-hand descriptions.
Architecture: Perception Plus Three Agents
OmniScientist adds a full sensory system on top of the research pipeline:
- Perception layer: extracts scientific evidence directly from heterogeneous raw data (images, signals, audio, video, 3-D structures, trajectories, tables, formulae, graphs) without relying on precomputed features.
- Ideation Agent: proposes research questions grounded in direct observation of the data (e.g., noticing asymmetric gravitational lensing in astronomy imagery rather than working from literature text alone).
- Experiment Agent: designs and runs experiments while continuously perceiving intermediate outputs—images, audio, video generated during execution—and adjusting the design accordingly.
- Writeup Agent: composes the final manuscript with access to the full evidence chain, citing specific image regions, audio timestamps, or 3D viewpoints.
- 36/36 cases: completed the full path from raw data to a compiled manuscript
- Mean overall paper score: 6.3 with the reference reasoning backbone
- Paired comparison vs. a 'blind' variant (receiving only precomputed scalar features): direct perception improved all 7 evaluation dimensions and won 85% of head-to-head judgments
- Perception remains computational; the seasoned scientist's intuitive 'this data looks odd' feeling is still human-only
- Creativity stays within existing paradigms—revolutionary paradigm-shifting discoveries remain out of reach
- A 6.3/10 score is promising but still short of top-tier journal acceptance; narrative craft and anticipating reviewers are human strengths
A deterministic pipeline lets observations shape research questions, experimental decisions, and final claims throughout the lifecycle. Three code-enforced checks ensure rigor:
1. Idea Check — novelty screening against existing literature 2. Rigour Check — statistical validity of experimental design 3. Claim Check — every conclusion traceable to raw data, with execution provenance and numerical traceability
Evaluation: 36 Real-Data Cases
The system was tested on 36 real-data cases spanning 5 discipline families (medicine/health, astronomy/earth science, physics/chemistry, life sciences, cognitive/neuroscience) and 4 families of scientific evidence.
Key results:
The contrast experiment is the central finding: even high-quality precomputed features cannot substitute for direct perception. Subtle, cross-modal, procedural information—the kind behind discoveries like anomalous dark-matter substructure hinted by asymmetric lensing—only emerges when the agent sees the raw data itself.
Limitations and Outlook
The forum post's commentary notes honest boundaries:
Reference
Li, B., Fei, H., Ju, T., Lee, M. L., & Hsu, W. (2026). OmniScientist: An Omni-Modal Omni-Discipline AI Scientist. *arXiv preprint arXiv:2608.13558*.