How Seemingly Inconsequential Design Choices Dictate LLM Performance on Whole-Slide Pathology Images

Forum topic · 小凯 · 2026-06-12

Summary

A new arXiv paper (2606.12407) by Weihrauch, Buckley, Lotter, and Manrai challenges the belief that general-purpose LLMs are inherently weak on whole-slide image (WSI) pathology tasks. Because WSIs exceed LLM context limits, prior evaluations typically used small, high-magnification patches processed independently with majority voting. The authors perform a systematic factorial analysis of four input design factors: inference mode, patch size, magnification, and patch count. They show that previous studies exaggerated the gap between specialized pathology models and generalist LLMs through suboptimal input configurations. On the MultiPathQA benchmark, switching to a single balanced configuration (low magnification, large patches, joint processing) improves GPT-5 accuracy from 15.1% to 39.5% on cancer type classification (TCGA) and from 38.1% to 62.9% on organ classification (GTEx). Per-task optimization pushes results further to 43.9% (TCGA) and 71.6% (GTEx). The same configuration generalizes to two additional models and a fully held-out CPTAC cohort, lifting Gemini 3 Flash by 23.4 percentage points without task-specific tuning. The findings suggest claims that domain-specific training is necessary for WSI pathology may partly reflect flawed baseline evaluation design.

Overview

Field: Computer Vision (Pathology) Authors: Kian R. Weihrauch, Thomas A. Buckley, William Lotter, Arjun K. Manrai Published: 2026-06-10 arXiv: 2606.12407

English Translation of the Abstract

General-purpose large language models (LLMs) are routinely used as baselines when evaluating specialized pathology models on whole-slide images (WSIs). Because WSIs exceed contemporary model context limits, LLM baselines typically use small, high-magnification patches processed independently via majority voting, without systematic evaluation of seemingly inconsequential design choices such as patch size, patch count, and magnification. Generalist LLMs have consistently underperformed specialized systems, reinforcing the perception that domain-specific training or architectural adaptation is necessary for pathology tasks involving WSIs.

This paper conducts a systematic factorial analysis of four input design factors: inference mode, patch size, magnification, and patch count. The authors demonstrate that prior studies exaggerated the gap between specialized models and generalist LLMs by choosing non-optimal input configurations.

Key Results

On the MultiPathQA benchmark, switching to a single balanced configuration (low magnification, large patches, joint processing) improves GPT-5:
Cancer type classification (TCGA): 15.1% → 39.5%
Organ classification (GTEx): 38.1% → 62.9%
Per-task optimization brings further gains up to 43.9% (TCGA) and 71.6% (GTEx).
The same configuration generalizes to two additional models and a fully held-out CPTAC cohort, boosting Gemini 3 Flash by 23.4 percentage points without any task-specific tuning.

Takeaway

What appears to be an inherent weakness of generalist LLMs on WSI pathology may largely stem from suboptimal evaluation design. Rigorous baseline configuration matters: patch size, magnification, patch count, and inference mode can dramatically change reported performance and conclusions about the need for domain-specific models.

--- *Auto-collected on 2026-06-12*