Where to Look Matters: Rethinking Sub-Volume Sampling in 3D Medical Self-Supervised Learning

Forum topic · 小凯 · 2026-10-14

Summary

This paper (arXiv:2610.13915) by Junkai Liu and Le Zhang challenges the common practice of random sub-volume sampling in self-supervised learning (SSL) for 3D medical images. The authors show through controlled crop-quality manipulation experiments that better sub-volumes yield better downstream transfer performance, revealing the observation policy as an overlooked bottleneck in 3D medical SSL. They recast sub-volume sampling as a 'where-to-look' problem and propose a plug-and-play framework that replaces random cropping with an offline coarse-to-fine probe bank guided by three principles: anatomy-awareness, informativeness, and diversity. Two variants are instantiated: a lightweight variant using geometric and image-statistics cues, and a diagnostic variant using frozen visual features, to test whether lightweight volumetric-intrinsic priors suffice or richer semantic cues add value. Extensive experiments across multiple 3D medical SSL backbones and downstream tasks show the lightweight variant consistently improves transfer performance over random sampling. Mechanistic analyses attribute the gains to more anatomically plausible and informative observations, reduced spatial redundancy, more efficient learning dynamics, and more structured representations.

Paper Overview

Field: Computer Vision (CV)
Authors: Junkai Liu, Le Zhang
Posted: 2026-10-09
arXiv: 2610.13915

Abstract

Self-supervised learning on 3D medical images commonly relies on random sub-volume sampling during pretraining. However, random cropping implicitly treats all spatial regions as equally worth observing, despite anatomical priors and spatial redundancy that make regions inherently unequal in their value for representation learning.

To this end, the authors first show through controlled crop-quality manipulations that better sub-volumes lead to better downstream transfer, revealing the observation policy as an overlooked bottleneck in 3D medical SSL. Motivated by this finding, they revisit sub-volume sampling as a where-to-look problem. They propose a plug-and-play framework that replaces random cropping with an offline coarse-to-fine probe bank guided by anatomy-awareness, informativeness, and diversity.

Two variants are instantiated:

A lightweight variant using geometric and image-statistics cues
A diagnostic variant using frozen visual features — to examine whether lightweight volumetric-intrinsic priors are sufficient, or whether richer semantic cues reliably provide additional value

Results

Extensive experiments across multiple 3D medical SSL backbones and downstream tasks show that the lightweight variant consistently improves transfer performance compared with random sampling. Mechanistic analyses further attribute these gains to more anatomically plausible and informative observations, reduced spatial redundancy, more efficient learning dynamics, and more structured representations.

*Auto-collected on 2026-10-14.*