Ranking-Aware Prompt Optimization for Multimodal Clinical Diagnosis (arXiv 2609.26730)

Forum topic · 小凯 · 2026-10-02

Summary

This paper introduces Ranking-PE (pair-level Pareto prompt evolution), a prompt optimization method for multimodal large language models (MLLMs) in clinical diagnosis. Standard adaptation pipelines optimize for accuracy, which is misleading under severe class imbalance common in clinical data; a constant majority-class predictor can exceed 90% accuracy while being clinically useless. The authors instead evaluate and optimize AUROC, a threshold-free ranking metric invariant to class balance. Building on reflective prompt optimization frameworks like GEPA, Ranking-PE replaces the binary per-instance correctness matrix with pairwise ranking rows over (positive, negative) instance pairs: a cell is 1 if the candidate prompt ranks the positive above the negative. By the Wilcoxon-Mann-Whitney identity, column means then equal empirical AUROC. The substitution is applied at all three points the search reads—the Pareto-dominance scoring matrix, per-instance feedback to the reflection LM, and final candidate selection—without extra model calls or surrogate losses. On three MIMIC diseases, accuracy-based prompt evolution can even worsen ranking, while Ranking-PE improves AUROC by +5.8 points on fine-tuned Qwen3-VL-8B and +16.2 points on MedGemma-4B.

Ranking-Aware Prompt Optimization for Multimodal Clinical Diagnosis

Field: NLP Authors: Tian Xia, Minghao Liu, Yiqing Liang, et al. Published: 2026-09-30 arXiv: 2609.26730

Overview (translated abstract)

Multimodal large language models (MLLMs) are rapidly advancing clinical diagnosis, yet their adaptation pipelines remain anchored to accuracy-based objectives. Clinical data are heavily class-imbalanced: a constant-majority predictor can score above 90% accuracy while being clinically useless. The authors therefore evaluate and optimize for AUROC, a threshold-free score that ranks positives above negatives and is invariant to class balance.

The work focuses on prompt optimization in MLLMs. Reflective methods such as GEPA use a binary scores matrix with one row per evaluation instance and one column per candidate prompt; cells record per-instance correctness, so the column average is accuracy, which drives candidate selection.

The proposed method, pair-level Pareto prompt evolution (Ranking-PE), replaces each correctness row with a pairwise ranking row over (positive, negative) instance pairs: a cell is 1 if the candidate prompt scores the positive above its paired negative. The column average then equals the empirical AUROC, guaranteed by the Wilcoxon-Mann-Whitney identity.

This substitution is applied at all three levels that the prompt-evolution search reads—the scoring matrix that determines Pareto dominance, the per-instance feedback provided to the reflection LM, and the final candidate selection—without additional model calls or a surrogate loss.

Results

On three diseases from the MIMIC dataset, accuracy-based prompt evolution can actually worsen ranking performance. Ranking-PE reverses this trend:

+5.8 AUROC percentage points over the accuracy-based approach on fine-tuned Qwen3-VL-8B
+16.2 AUROC percentage points on MedGemma-4B

Original abstract (excerpt)

> Multimodal large language models (MLLMs) are rapidly advancing clinical diagnosis, yet their adaptation pipelines remain anchored to accuracy-based objectives. Clinical data are heavily class-imbalanced: a constant-majority predictor can score above 90% accuracy while being clinically useless. We therefore evaluate and optimize for AUROC, a threshold-free score that ranks positives above negatives and is invariant to class balance. We focus on prompt optimization in MLLMs. Reflective methods such as GEPA use a binary scores matrix with one row per evaluation instance and one column per candidate prompt; cells record per-instance correctness, so the column average is accuracy and drives candidate selection. We introduce pair-level Pareto prompt evolution (Ranking-PE), which replaces each co...

--- *Auto-collected on 2026-10-02*