BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models

Forum topic · 小凯 · 2026-09-11

Summary

BrainTaskonomy (arXiv:2609.10518) proposes organizing both pretraining and adaptation of fMRI foundation models using measured learning relations, without modifying the backbone. During pretraining, a lightweight Brain-DiT proxy estimates domain difficulty and directed facilitation across ten fMRI domains, producing a priority-guided cumulative domain curriculum combined with high-to-low-noise timestep scheduling and joint consolidation. During adaptation, controlled first- and higher-order transfer experiments across fifteen tasks build a directed taskonomy, from which budgeted integer programming (BIP) selects directly supervised source tasks and target-specific transfer routes. The joint priority-domain and high-to-low-timestep curriculum reduces v-NMSE, PSD-NMSE, and FC-MSE by 6.5%, 16.3%, and 10.5% versus uniform sampling, with strong downstream performance on six in- and out-of-domain tasks. The taskonomy reveals asymmetric, target-dependent transfer, and BIP policies show larger gains when higher-order route spaces are available. Authors: Junfeng Xia, Wenhao Ye, Junxiang Zhang, Jiayu Zuo, Mo Wang, Quanying Liu.

Paper Overview

Field: CV Authors: Junfeng Xia, Wenhao Ye, Junxiang Zhang, Jiayu Zuo, Mo Wang, Quanying Liu Published: 2026-09-09 arXiv: 2609.10518

Introduction

fMRI foundation models increasingly aggregate heterogeneous data across brain states, cohorts, and acquisition settings. However, pretraining domains are typically treated as a flat mixture, and downstream tasks are adapted independently. This paper investigates whether measured learning relations can organize both stages—without modifying the backbone architecture.

Method

Pretraining: Priority-Guided Domain Curriculum

A lightweight Brain-DiT proxy estimates two quantities across ten fMRI domains: difficulty and directed facilitation.
These estimates yield a priority-guided cumulative domain curriculum.
This is combined with a high-to-low-noise timestep scheduling and joint consolidation.

Adaptation: Directed Taskonomy

Controlled first- and higher-order transfer experiments across fifteen tasks construct a directed taskonomy.
Budgeted integer programming (BIP) selects directly supervised source tasks and target-specific transfer routes from this taskonomy.

Results

The joint priority-domain and high-to-low-timestep curriculum reduces:
v-NMSE by 6.5%
PSD-NMSE by 16.3%
FC-MSE by 10.5%
(relative to uniform sampling over both dimensions)
Strong downstream performance is demonstrated across six in- and out-of-domain tasks.
The taskonomy reveals asymmetric, target-dependent transfer.
Exploratory sealed-test evaluation shows larger descriptive gains for BIP policies when higher-order route spaces are available, compared to matched random controls.

Conclusion

The findings support organizing fMRI pretraining and adaptation by measured learning relations, rather than treating domains and tasks as independent flat sets.

--- *Source: arXiv:2609.10518*