Prediction Fragmentation Controls Test-Time Adaptation in Medical Image Segmentation

Forum topic · 小凯 · 2026-09-20

Summary

This paper addresses episodic test-time adaptation (TTA) for image segmentation, where a frozen model is reset to source weights and adapted for a fixed number of steps per case. The authors show that a fixed adaptation horizon conflates a cohort-level question (how far to adapt) with a per-case question (whether to adapt at all). On cross-vendor cardiac MRI, mean Dice improvement is statistically indistinguishable from zero while 58.7% of cases worsen individually. They quantify harm via harmful accepted area (HA) and propose prediction fragmentation—the geometric divergence between source and adapted masks—as a label-free predictor of HA. It achieves Spearman rho 0.50-0.60 across three benchmarks at one quarter the latency of gradient norms. A case-level router built on it cuts HA from 0.228 to 0.139 on an unseen benchmark and from 0.129 to 0.013 on cardiac data with comparable Dice, reducing the harmful-case rate from 58.7% to 20.0%. The approach requires no labels or gradients at decision time and transfers across architectures and domains.

Paper Overview

Field: Computer Vision Authors: Lili Wang, Jing Li, Xiaowen Sun, Xiangyu Hu, Zhuangzhuang Gu, Jian Liu, Srihari Nelakuditi, Yan Tong Published: 2026-09-17 arXiv: 2609.20700

Summary (Chinese)

回合式测试时自适应（episodic TTA）在每个病例上把冻结分割器重置回源权重 M0，再以固定步数自适应。固定视界把两个不同问题混为一谈：群体层面的“该自适应多远”与本质上逐病例的“该病例是否应自适应”。群体均值掩盖了这一决策：跨厂商心脏 MRI 上，自适应带来的平均 ΔDice 与零统计上不可区分，而 58.7% 的病例个体上反而变差。我们把这种伤害量化为有害接受面积（HA）——控制器部署的有害编辑面积占比。留出调参给出的基线强于固定视界，但其选择的预算在两个主要医学基准上都不可迁移，且任何全局预算都无法以病例为条件。我们证明：预测碎片化——M0 与自适应掩膜 Mk 之间的分歧几何——无需标签、决策时无需额外反向传播即可预测 HA，在三个基准上表现相当（Spearman ρ 0.50–0.60），延迟仅为梯度范数的四分之一。基于它的病例级路由器把某设计外基准上的 HA 从 0.228 降到 0.139；在其设计所选的心脏基准上，HA 从 0.129 降到 0.013（Dice 相当、平均仅 1.10 次部署更新），并把 58.7% 的有害率降到 20.0%。阈值在与评估不相交的有标注划分上拟合一次；决策不用标签与梯度。该模板可跨架构与领域移植（nnU-Net→SegFormer，Cityscapes→ACDC）。

Abstract (Original)

Episodic test-time adaptation resets a frozen segmenter to source weights $$M_0$$ on each case and adapts for a fixed step count. A fixed horizon conflates a cohort-level question, how far to adapt, with an irreducibly per-case one, whether this case should be adapted at all. Cohort means hide that decision: on cross-vendor cardiac MRI the mean $$Δ$$ Dice from adaptation is statistically indistinguishable from zero while 58.7% of cases are individually made worse. We quantify this harm as harmful accepted area (HA), the harmful fraction of the edited area a controller deploys. Held-out tuning gives a stronger baseline than a fixed horizon, but the budget it selects transfers on neither of the two main medical benchmarks, and no global budget can condition on the case. We show that prediction fr...

--- *Auto-collected on 2026-09-20*