Paper Overview
- Research area: Computer Vision (CV)
- Authors: Emre Kutlusan, Yongmin Li
- Published: 2026-11-10
- arXiv: 2611.14519
- Adapted foundation models generalize better on all external datasets.
- Within the training distribution, task-specific models remain competitive when sufficient data is available and achieve finer lesion boundary delineation; foundation models are more prone to missing small lesions.
- Foundation models only require adapting a small fraction of their capacity; the required capacity is determined by the imaging domain rather than by the adaptation method.
- The authors discovered image duplication between a widely used polyp test set and another benchmark.
Summary
Foundation models pretrained on large natural-image collections are increasingly adapted for medical image segmentation, with the expectation that they will replace models trained for each individual task. However, most prior comparisons rely on data from the training distribution and a single task-specific baseline whose configuration is rarely justified, providing limited evidence of performance at new clinical sites. It also remains unclear how much of a foundation model must be adapted, and what determines this.
The authors adapted DINOv2 with strategies ranging from tuning only its bias terms to full fine-tuning, and compared it with U-Nets and ResU-Nets trained from scratch. Experiments covered colonoscopy, breast ultrasound, and dermoscopy, with testing on both held-out data from the training source and independent datasets from other institutions.
Key Findings
Conclusion
Adapting foundation models is worthwhile for cross-site clinical scenarios, and the adaptation capacity should be selected based on the target imaging domain.