ClinHallu: A Benchmark for Stage-Wise Hallucination Diagnosis in Medical Multimodal LLMs

Forum topic · 小凯 · 2026-06-16

Summary

ClinHallu is a new benchmark for diagnosing where hallucinations originate in medical multimodal large language model (MLLM) reasoning, presented in arXiv paper 2606.14697 by Sicheng Yang, Hangjie Yuan, and Wenjun Zhang. Unlike existing medical hallucination benchmarks that focus mainly on data collection, ClinHallu enables source-level diagnosis by decomposing reasoning into three stages: visual recognition, knowledge recall, and reasoning integration. The benchmark contains 7,031 validated instances, each augmented with a structured reasoning trace. The authors apply stage-substitution interventions to measure how correcting a specific stage affects the final answer, and show that trajectory-supervised fine-tuning can reduce stage-wise hallucinations. ClinHallu provides a fine-grained testbed for diagnosing and mitigating reasoning failures in medical MLLMs, with the dataset publicly available on GitHub under alibaba-damo-academy.

Paper Overview

Field: Computer Vision (CV) Authors: Sicheng Yang, Hangjie Yuan, Wenjun Zhang Posted: 2026-06-12 arXiv: 2606.14697

Abstract

Building trustworthy medical multimodal large language models (MLLMs) is critical for reliable clinical decision support. Existing medical hallucination benchmarks mainly focus on data collection, but often ignore where hallucinations originate within the reasoning process. The authors find that hallucination sources vary across samples: errors may arise from visual misrecognition, incorrect medical knowledge recall, or flawed reasoning integration.

To enable source-level hallucination diagnosis, the paper introduces ClinHallu, a benchmark for stage-wise hallucination diagnosis in medical MLLM reasoning.

Key Features

7,031 validated instances, each augmented with a structured reasoning trace decomposed into three stages:
Visual Recognition
Knowledge Recall
Reasoning Integration
Stage-substitution interventions: used to measure how correcting a specific stage affects the final answer.
Trajectory-supervised fine-tuning: demonstrated to reduce stage-wise hallucinations beyond evaluation alone.

Conclusion

ClinHallu provides a fine-grained hallucination testbed for diagnosing and mitigating reasoning failures in medical MLLMs. The benchmark is publicly available at:

https://github.com/alibaba-damo-academy/ClinHallu