How Seemingly Inconsequential Design Choices Dictate Performance in LLM-Based Pathology

Forum topic · 小凯 · 2026-06-12

Summary

A new arXiv paper (2606.12407) shows that trivial input design choices—not model capability alone—explain much of the gap between generalist LLMs and specialized pathology models on whole-slide images (WSIs). Because WSIs exceed context limits, LLM baselines typically process small, high-magnification patches independently with majority voting, without systematically evaluating patch size, patch count, magnification, or inference mode. The authors perform a factorial analysis of these four factors and find prior studies used suboptimal configurations, overstating the gap. On MultiPathQA, switching to a single balanced configuration (large patches, low magnification, joint processing) raises GPT-5 from 15.1% to 39.5% on TCGA cancer-type classification and from 38.1% to 62.9% on GTEx organ classification; per-task tuning reaches 43.9% and 71.6%. The same configuration transfers to two other models and a fully held-out CPTAC cohort, boosting Gemini 3 Flash by 23.4 points without task-specific tuning.

Overview

Field: Computer Vision / Computational Pathology Authors: Kian R. Weihrauch, Thomas A. Buckley, William Lotter, Arjun K. Manrai arXiv: 2606.12407

Abstract

General-purpose large language models (LLMs) are routinely used as baselines when evaluating specialized pathology models on whole-slide images (WSIs). Because WSIs exceed contemporary model context limits, LLM baselines typically use small, high-magnification patches processed independently via majority voting, without systematic evaluation of seemingly inconsequential design choices such as patch size, patch count, and magnification. Generalist LLMs have consistently underperformed specialized systems, reinforcing the perception that domain-specific training or architectural adaptation is necessary for pathology tasks involving WSIs.

This paper conducts a systematic factorial analysis of four input design factors: inference mode, patch size, magnification, and patch count. The authors demonstrate that prior studies exaggerated the gap between specialized models and generalist LLMs by selecting non-optimized input configurations.

Key Results

On the MultiPathQA benchmark, switching to a single balanced configuration — large patches at low magnification, processed jointly — improves GPT-5:

Cancer-type classification (TCGA): 15.1% → 39.5%
Organ classification (GTEx): 38.1% → 62.9%

Per-task optimization further improves performance to up to 43.9% (TCGA) and 71.6% (GTEx).

The same configuration generalizes to two additional models and a fully held-out CPTAC cohort, improving Gemini 3 Flash by 23.4 percentage points without any task-specific tuning.

Takeaway

Before concluding that domain-specific training or architectural adaptation is required for WSI pathology tasks, LLM baselines should be evaluated under properly optimized input configurations — the perceived capability gap may be largely an artifact of suboptimal design choices.