OmniScientist: An Omni-Modal Omni-Discipline AI Scientist

Paper: OmniScientist: An Omni-Modal Omni-Discipline AI Scientist Authors: Bobo Li, Hao Fei, Tianjie Ju, Mong-Li Lee, Wynne Hsu arXiv ID: 2608.13558v1 (cs.AI, cs.CL)

The Problem: "Blind" AI Scientists

Recent AI scientist systems can automate research workflows—from hypothesis generation and code execution to manuscript preparation. But like a detective who can only read text reports and never inspect photos, recordings, or fingerprints at the crime scene, existing systems reason only over text, code, labels, or precomputed scalar summaries. Scientifically decisive spatial, temporal, cross-channel, and procedural relations remain invisible to them.

History shows why this matters: Watson and Crick's discovery of the DNA double helix came from *perceiving* X-ray diffraction patterns (Photo 51), not from text summaries. Major scientific insights often begin with direct perception of raw multimodal data—an astronomer noticing an unusual halo, a biologist spotting anomalous protein trajectories, a neuroscientist hearing an unfamiliar rhythm in EEG audio.

Architecture: A Full Sensory System for Research

OmniScientist is an end-to-end, omni-modal AI scientist that conducts multidisciplinary research directly from heterogeneous raw evidence:

• Perception layer: Extracts scientific evidence directly from raw data—images, signals, audio, video, 3-D structures, trajectories, tables, formulae, and graphs—without relying on precomputed summaries or scalar features.
• Three autonomous agents operating in a deterministic pipeline:
• Ideation Agent: Observes raw data directly, then poses research questions grounded in what it perceives (e.g., asymmetric gravitational lensing in galaxy cluster images).
• Experiment Agent: Designs experiments, writes code, and continuously perceives intermediate results—adjusting experimental decisions based on process-level observations, not just final metrics.
• Writeup Agent: Writes papers with direct access to the full evidence chain, citing specific image regions, audio timestamps, and 3-D viewpoints.

Because observations shape research questions, experimental decisions, and final claims across the whole lifecycle, discoveries remain evidence-grounded.

Three Code-Enforced Checks

To guard against false positives, p-hacking, and irreproducibility, all checks run in code:

1. Idea Check — verifies novelty via literature retrieval. 2. Rigour Check — validates statistical validity (sample sizes, controls, statistical tests). 3. Claim Check — traces every conclusion back to original data, ensuring execution provenance and numerical traceability.

Evaluation: 36 Real-Data Cases Across 5 Disciplines

OmniScientist was tested on 36 real-data cases spanning:

• Medicine and health sciences (pathology images, ECG signals, CT 3-D reconstruction)
• Astronomy and Earth sciences (galaxy images, seismic signals, weather prediction)
• Physics and chemistry (molecular dynamics, crystal structures, spectra)
• Life sciences (protein structures, gene expression time series, cell trajectories)
• Cognitive and neuroscience (EEG signals, behavior videos, neuroimaging)

Results:

• Completed the full path from raw data to a compiled manuscript in all 36 cases.
• Achieved a mean overall paper score of 6.3 with the reference reasoning backbone.
• In paired comparisons against a blind variant (receiving only precomputed scalar features), direct perception improved all 7 evaluation dimensions and won 85% of head-to-head judgments.

Key Takeaways

• Perception beats precomputation: Even the highest-quality precomputed features cannot replace direct perception of raw data. The subtle, cross-modal, procedural information that drives discovery is only capturable through direct sensing.
• Self-auditing science: By enforcing novelty screening, statistical validity, and claim traceability in code, the system can review its own work.

Limitations and Outlook

The authors and the original commentary note honest boundaries: OmniScientist's perception is still computational and lacks a veteran scientist's intuition; its creativity operates largely within existing paradigms rather than breaking them; and a 6.3 mean score still falls short of top-journal acceptance standards. Still, the work marks a clear direction: from "text reasoners" to "multimodal perceivers"—with future capabilities in auditory sensing, visual-temporal reasoning, structural intuition, and cross-modal correlation. Like the microscope for biologists, such systems are positioned to be powerful assistants rather than replacements for human scientists.

Reference

Li, B., Fei, H., Ju, T., Lee, M. L., & Hsu, W. OmniScientist: An Omni-Modal Omni-Discipline AI Scientist. *arXiv preprint* arXiv:2608.13558.