Paper Overview
Field: NLP Authors: Tian Xia, Minghao Liu, Yiqing Liang, et al. Published: 2026-09-30 arXiv: 2609.26730
Summary
Multimodal large language models (MLLMs) are rapidly advancing clinical diagnosis, yet their adaptation pipelines remain anchored to accuracy-based objectives. Clinical data are heavily class-imbalanced: a constant-majority predictor can score above 90% accuracy while being clinically useless. The authors therefore evaluate and optimize for AUROC, a threshold-free metric that ranks positives above negatives and is invariant to class balance.
Method: Ranking-PE
The work focuses on prompt optimization in MLLMs. Reflective methods such as GEPA use a binary scores matrix with one row per evaluation instance and one column per candidate prompt; cells record per-instance correctness, so the column average is accuracy, which drives candidate selection.
Ranking-PE (pair-level Pareto prompt evolution) replaces each correctness row with a pairwise ranking row over (positive, negative) instance pairs: a cell is 1 if the candidate prompt scores the positive above the paired negative. The column average then equals the empirical AUROC, guaranteed by the Wilcoxon-Mann-Whitney identity.
The substitution is applied at all three levels that prompt evolution search reads:
- The scores matrix that determines Pareto dominance
- Per-instance feedback provided to the reflective LM
- Final candidate selection
- +5.8 AUROC points over accuracy-based optimization on fine-tuned Qwen3-VL-8B
- +16.2 AUROC points on MedGemma-4B
This requires no additional model calls and introduces no surrogate losses.
Results
On three MIMIC diseases, accuracy-based prompt evolution can actually worsen ranking performance. Ranking-PE reverses this trend:
*Auto-collected on 2026-10-02.*