MARCUS: When AI Learns to Read the Heart's Three Languages
A deep-dive translation and summary of a Chinese forum post explaining the MARCUS cardiac diagnosis AI system (originally published on zhichai.net).
Overview
Cardiovascular disease is the world's leading cause of death (~18 million deaths per year, 32% of all deaths per WHO data). Diagnosis relies on three complementary but incomplete windows into the heart:
- ECG — records electrical activity; cheap and fast, but blind to structure
- Echocardiogram — real-time ultrasound video of chambers and valves; operator-dependent
- Cardiac MRI (CMR) — the most precise structural imaging; expensive and time-consuming
- Architecture: MARCUS uses a hierarchical agentic design. Three specialist vision-language models (ECG, echocardiography, CMR experts) each handle their own modality; a Multimodal Orchestrator decomposes complex questions, dispatches sub-questions to specialists, integrates findings, and resolves conflicts between modalities.
- Vision encoders: A SigLIP-based encoder for ECG waveforms and multi-view, temporally-aware encoders for echo and CMR video. These are trained from scratch on medical data rather than reusing internet-image encoders.
- Core model: A 3-billion-parameter vision-language model serves as the reasoning and language backbone.
- Domain-specific visual encoders trained on millions of cardiac images learn modality-specific patterns (ST elevation, valve motion, scar tissue).
- Temporal aggregation modules understand "cardiac movies," not just single frames — critical for assessing contractile function.
- Cross-view fusion integrates multiple ultrasound/CMR views (parasternal long/short axis, apical chambers) into a coherent picture.
- GRPO reinforcement learning shifts the model from pattern-matching to genuine reasoning with explainable rationales.
- Orchestrator coordination lets specialists stay focused while enabling integrative, scalable diagnosis (new modalities can be added as new specialists).
- Emergency triage — pre-screening chest-pain ECGs before a physician arrives
- Primary care empowerment — specialist-level echo interpretation at resource-limited hospitals
- Complex case consultation and medical education
- Data bias: training data from only two US hospitals may not generalize across populations
- Rare diseases: extremely rare conditions remain under-represented
- Causality: MARCUS captures associations, not necessarily pathophysiological mechanisms
- Liability: responsibility for AI-assisted diagnostic errors remains legally unresolved
- Integrating more modalities (coronary CT, nuclear imaging, genomics) toward a full "digital cardiac twin"
- Prospective clinical trials to validate impact on patient outcomes
- Personalized diagnostics incorporating history, genetics, and lifestyle
- Improved explainability (visual attention maps aligned with clinician reasoning)
- Paper: arXiv:2603.22179 — O'Sullivan JW, et al. "MARCUS: An agentic, multimodal vision-language model for cardiac diagnosis and management"
- Code: https://github.com/AshleyLab/MARCUS
Human experts who can integrate all three are scarce, and inter-reader agreement among cardiologists can be as low as 60-70%. MARCUS (Multimodal Autonomous Reasoning and Chat for Ultrasound and Signals), a 2026 arXiv paper from Stanford (arXiv:2603.22179), proposes an AI system that reads all three "languages" of the heart simultaneously.
Key points
Training pipeline (three stages)
1. Visual pretraining — each specialist learns to generate radiologist-style descriptions (language model frozen to preserve general language ability). 2. Supervised fine-tuning — 741,000 visual question-answer pairs (460K ECG, 155K echo, 126K CMR). 3. Reinforcement learning — Group Relative Policy Optimization (GRPO) over 879,000 multiple-choice questions, teaching step-by-step diagnostic reasoning rather than memorized answers.
Training data scale
| Data | Quantity | |---|---| | ECG recordings | 250,000 | | Echo images | 1.27 million (10,823 studies) | | CMR images | 12.19 million (9,473 studies) | | VQA pairs | 741,000 | | RL questions | 879,000 | | Total | ~16 million samples |
All data comes from real clinical exams at Stanford Hospital and UCSF, each paired with attending-physician reports as ground truth.
Benchmark results
Unimodal interpretation (accuracy)
| Modality | MARCUS (Stanford) | MARCUS (UCSF) | Frontier models (GPT-5 / Gemini 2.5 Pro) | |---|---|---|---| | ECG | 87% | 91% | 35-48% | | Echocardiography | 67.4% | 86.0% | 24-35% | | CMR | 88% | 85% | 47-58% |
Multimodal integration
On cases requiring combined ECG + echo + CMR analysis, MARCUS reaches 70% accuracy vs. 22-28% for GPT-5 Thinking and Gemini 2.5 Pro Deep Think — roughly a 3x gap (P<0.001 for unimodal differences).
Why it wins
"Mirage Reasoning" and hallucination resistance
The paper describes "mirage reasoning": vision-language models deriving answers from incidental text embedded in images (e.g., printed labels on an ECG) rather than the actual waveforms. Frontier models are notably susceptible; MARCUS's specialist architecture, clear task boundaries, and large-scale medical training make it substantially more resistant.
Clinical implications
Potential use cases include:
The authors announce plans to open-source the model, code, and benchmarks (GitHub: https://github.com/AshleyLab/MARCUS), which could make MARCUS a community baseline for medical AI, similar to ResNet or BERT in general vision/language research.