Paper Overview
- Research Area: NLP
- Authors: Bobo Li, Hao Fei, Tianjie Ju, Mong-Li Lee, Wynne Hsu
- Release Date: 2026-08-13
- arXiv: 2608.13558
- Problem: Existing AI scientist systems rely on text, code, labels, or precomputed summaries, making scientifically decisive spatial, temporal, cross-channel, and procedural relations unavailable to the agent.
- Solution: OmniScientist, an end-to-end omni-modal AI scientist with a perception layer and three autonomous agents (ideation, experiment, writeup) running inside a deterministic pipeline.
- Deterministic checks: Novelty screening, statistical validity, execution provenance, and numerical traceability are enforced by running idea, rigor, and claim checks directly in code.
- Evaluation breadth: 36 real-data cases across 5 discipline families and 4 evidence families, covering modalities such as images, signals, audio, video, 3D structures, trajectories, tables, formulae, and graphs.
- Results:
- Completed the full path from raw data to compiled manuscript in all 36 cases.
- Mean overall paper score of 6.3 with the reference reasoning backbone.
- Direct perception improved all 7 evaluation dimensions versus a blind variant receiving only precomputed scalar features.
- Won 85% of head-to-head judgments against the blind variant.
- Implication: Lifecycle-wide perception is essential for evidence-grounded scientific discovery and points toward broadly capable AI scientist systems.
- arXiv: 2608.13558
Chinese Summary
Recent advances in foundation models have enabled AI scientists to automate increasingly complete research workflows, from hypothesis generation and code execution to manuscript preparation. However, workflow coverage alone does not provide access to the full evidence on which scientific discovery depends. Existing systems typically reason over text, code, labels, or precomputed summaries, leaving scientifically decisive spatial, temporal, cross-channel, and procedural relations invisible to the agent. We propose OmniScientist, an end-to-end omni-modal AI scientist that conducts multidisciplinary research directly from heterogeneous raw evidence. A perception layer and three autonomous agents (ideation, experiment, and writeup) operate within a deterministic pipeline, enabling observations to shape research questions, experimental decisions, and final claims throughout the research lifecycle. By running idea, rigor, and claim checks in code, the system enforces novelty screening, statistical validity, execution provenance, and numerical traceability. We evaluate OmniScientist on 36 real-data cases spanning 5 discipline families, 4 families of scientific evidence, and modalities including images, signals, audio, video, 3D structures, trajectories, tables, formulae, and graphs. The system completes the full path from raw data to compiled manuscript in all 36 cases and achieves a mean overall paper score of 6.3 with the reference reasoning backbone. In paired comparisons against a blind variant receiving only precomputed scalar features, direct perception improves all 7 evaluation dimensions and wins 85% of head-to-head judgments. These results show that lifecycle-wide perception is essential for evidence-grounded scientific discovery and provide a practical path toward broadly capable AI scientists.