CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs — Paper Explained

> Paper: CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs > Authors: Hashmat Shadab Malik, Anees Ur Rehman Hashmi, Numan Saeed, Muzammal Naseer, Salman Khan > arXiv: 2606.27264 > Field: cs.CV

The Problem: Diagnoses Without Reasons

When an AI model says a lung nodule is malignant but cannot explain why, its stated rationale may be fabricated — the hallucination problem. Existing evaluation only checks the final answer (benign vs. malignant), leaving the diagnostic process a black box. A model can reach a correct conclusion through entirely wrong reasoning, cite imaging features that do not exist, or force irrelevant regions to support its conclusion.

Current medical QA datasets make this worse: they compress expert radiology reports into simple Q&A pairs. A report describing an irregular, spiculated, lobulated 1.2 cm × 0.8 cm nodule with heterogeneous enhancement becomes "Q: benign or malignant? A: malignant." The evidence is discarded; only the verdict survives. The paper also notes that 3D CT is harder than 2D X-rays because diagnosis requires spatial reasoning about how structures relate in three dimensions.

The Solution: A Four-Stage Reasoning Pipeline

CORTEX structures the radiologist's diagnostic process into four explicit, auditable stages:

1. Task Understanding — state the clinical context (patient history, screening vs. diagnosis), since the same image reads differently for a 60-year-old smoker vs. a 20-year-old non-smoker. 2. Visual Observation — collect concrete, localizable evidence: structure, size, shape, density, margins, location (e.g., "15 mm solid nodule, right upper lobe posterior segment, lobulated margins, spiculation"), not vague statements like "lung abnormality." 3. Diagnostic Reasoning — link observations to medical knowledge step by step, perform differential diagnosis, weigh evidence strength, and acknowledge uncertainty. No skipping steps. 4. Answer Synthesis — produce a conclusion consistent with the reasoning, with an explicit confidence level ("70% malignancy probability"), recommendations, and alternative diagnoses. Inconsistency between reasoning and conclusion is flagged by the evaluation system.

Dataset Construction and Evaluation

• Built on the public CT-RATE chest CT dataset.
• Frontier LLMs (e.g., GPT-4) generate reasoning trajectories; expert radiologists verify them, with iterative correction.
• Final corpus: 76,177 validated reasoning trajectories across open-ended VQA, closed-ended VQA, and report generation.
• Evaluation uses stage-level rubric scoring (points for correctly identifying nodule location, size, margins, etc.) followed by human expert review of logic, medical correctness, and completeness.

Key Points

• Structure forces reasoning: without structured supervision, models learn pattern matching ("circular shape → answer is 3.14") rather than genuine diagnostic logic. CORTEX makes high scores impossible without correct observation → knowledge linkage → consistent conclusion.
• Verifiability over accuracy alone: in medicine, trustworthiness means knowing *where* a model failed. If a model claims spiculated margins, a clinician can check the image; absent features invalidate the diagnosis.
• White-box AI: CORTEX does not open network weights but makes the *reasoning process* visible and independently checkable, so errors can be corrected, learned from, and prevented.
• Cognitive scaffolding: the four-stage structure mirrors how humans learn complex tasks — frameworks that ensure no critical thinking step is skipped, trading some speed for systemic correctness.
• Uncertainty as a feature: models must express calibrated confidence. A 70% malignancy probability suggests biopsy; 95% suggests surgery. Learning to say "I don't know" is treated as valuable, echoing how clinicians internalize epistemic uncertainty.
• Assistant, not replacement: a transparent AI becomes "here is my analysis, check it" rather than "trust me" — a role more aligned with medical ethics, where the physician's final judgment remains irreplaceable.

Takeaway

Medical diagnosis is fundamentally a narrative: weaving scattered imaging observations into a logical, evidenced story. Traditional AI diagnosis acts as an auto-summarizer that compresses the image into a verdict. CORTEX instead builds a narrative framework that forces AI to reason like a real storyteller — with an account of every step — so that in the best case, AI serves as the clinician's story partner, and the human decides whether the story is worth believing.

Reference

> Malik, H. S., Hashmi, A. U. R., Saeed, N., Naseer, M., & Khan, S. (2026). CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs. *arXiv preprint arXiv:2606.27264*.