Paper Overview
Field: NLP Authors: Iryna Hartsock, Cesar Lam, Christopher Otteni, Aliya Qayyum, Robert Gatenby, Cyrillo Araujo, Ghulam Rasool Posted: 2026-08-18 arXiv: 2608.18072
Abstract
Purpose: To develop and evaluate a locally deployed multi-agent AI system for radiology report structuring and quality assurance.
Materials and Methods: This retrospective study included 638 radiology reports from CT examinations of the chest, abdomen, and pelvis dictated by 15 board-certified radiologists in 2023 and 2024. A multi-agent AI pipeline was developed to perform report structuring and quality assurance (QA). The system structured the report into standardized anatomical sections at the sentence level using regex rules and local large language models. It also detected mismatches between the Findings and Impression sections, or within sections; gender-anatomy conflicts; and undocumented communication of critical findings. Two board-certified radiologists independently evaluated a 45-report subset.
Results: The multi-agent system structured the Findings sections of all reports (22,270 sentences) into predefined anatomical formats while preserving the original report content. The system flagged 90 (14.1%) reports, most commonly for section mismatches (80 reports, 12.5%). In radiologist evaluation, both reviewers agreed that 31 (69%) were correctly restructured and 2 (4%) were incorrectly restructured, and they disagreed on the remaining 12 (27%). Both reviewers agreed that no clinically important information was omitted and no fabricated content was introduced. Overall QA performance was rated "excellent" or "good" in 84% of evaluated reports; the remainder were rated "fair".
Conclusion: The locally deployed multi-agent AI system combines radiology report structuring and quality assurance into a single workflow. The system showed favorable performance in radiologist evaluation. Such systems could support report standardization and quality assurance in radiology practice.
--- *Auto-collected on 2026-08-20.*