Overview
A forum post discussing the arXiv paper 2608.18072 — a multi-agent AI system for radiology report structuring and quality assurance.
- Field: NLP
- Authors: Iryna Hartsock, Cesar Lam, Christopher Otteni, Aliya Qayyum, Robert Gatenby, Cyrillo Araujo, Ghulam Rasool
- Link: arXiv:2608.18072
- Purpose: Develop and evaluate a locally deployed multi-agent AI system for radiology report structuring and quality assurance (QA).
- Materials and methods: Retrospective study of 638 radiology reports from chest, abdomen, and pelvis CT examinations dictated by 15 board-certified radiologists in 2023–2024. The pipeline structures reports into standardized anatomical sections at the sentence level using regex rules and local large language models. It also detects mismatches between the Findings and Impression sections (or within sections), gender-anatomy conflicts, and undocumented communication of critical findings. Two board-certified radiologists independently evaluated a 45-report subset.
- Results:
- All reports' Findings sections (22,270 sentences) were structured into the predefined anatomical format while preserving original report content.
- 90 reports (14.1%) were flagged, most commonly for section mismatches (80 reports, 12.5%).
- In radiologist evaluation, both reviewers agreed 31 reports (69%) were correctly restructured and 2 (4%) incorrectly restructured, disagreeing on the remaining 12 (27%).
- Both reviewers agreed no clinically important information was omitted and no fabricated content (hallucinations) was introduced.
- Overall QA performance was rated "excellent" or "good" in 84% of evaluated reports; the rest were rated "fair".
- Conclusion: The locally deployed multi-agent AI system combines report structuring and quality assurance into a single workflow with favorable performance in radiologist evaluation, potentially supporting standardization and QA in radiology practice.