Generative Drifting (GDM): One-Step Conditional 3D Medical Image Generation

Forum topic · 小凯 · 2026-04-23

Summary

Researchers propose GDM, a generative drifting framework for conditional medical image generation, addressing the fundamental trade-off between inference efficiency, patient-specific fidelity, and distribution-level plausibility in high-dimensional 3D medical imaging. GDM reformulates deterministic medical image prediction as a multi-objective learning problem that jointly improves distribution-level plausibility and patient-specific fidelity while retaining single-step inference. The framework extends drifting to 3D medical imaging via an attractive-repulsive drift that minimizes discrepancy between the generator pushforward and the target distribution. For stable drifting-based learning on volumetric data, GDM builds multi-level feature banks from a medical foundation encoder, enabling reliable affinity estimation and drift field computation across complementary global, local, and spatial representations. A gradient reconciliation strategy in a shared output space balances competing distribution-level and fidelity-oriented objectives. Evaluated on MRI-to-CT synthesis and sparse-view CT reconstruction, GDM consistently outperforms GAN, flow matching, SDE-based generative models, and supervised regression baselines, improving anatomical fidelity, quantitative reliability, perceptual realism, and inference efficiency.

Paper Overview

Field: Computer Vision Authors: Zirong Li, Siyuan Mei, Weiwen Wu, Andreas Maier, Lina Gölz, Yan Xia Published: 2026-04-21 arXiv: 2604.19736

Original Abstract

Conditional medical image generation plays an important role in many clinically relevant imaging tasks. However, existing methods still face a fundamental challenge in balancing inference efficiency, patient-specific fidelity, and distribution-level plausibility, particularly in high-dimensional 3D medical imaging. In this work, we propose GDM, a generative drifting framework that reformulates deterministic medical image prediction as a multi-objective learning problem to jointly promote distribution-level plausibility and patient-specific fidelity while retaining one-step inference. GDM extends drifting to 3D medical imaging through an attractive-repulsive drift that minimizes the discrepancy between the generator pushforward and the target distribution. To enable stable drifting-based learning on 3D volumetric data, GDM constructs multi-level feature banks from a medical foundation encoder to support reliable affinity estimation and drift field computation across complementary global, local, and spatial representations. In addition, a gradient reconciliation strategy in a shared output space improves the optimization balance among competing distribution-level and fidelity-oriented objectives. We evaluate the proposed framework on two representative tasks: MRI-to-CT synthesis and sparse-view CT reconstruction. Experimental results show that GDM consistently outperforms a broad range of baselines, including GAN-based, flow matching, and SDE-based generative models as well as supervised regression methods, while improving the balance between anatomical fidelity, quantitative reliability, perceptual realism, and inference efficiency. These findings suggest that GDM provides a practical and effective framework for conditional 3D medical image generation.

Key Points

Problem: Balancing inference efficiency, patient-specific fidelity, and distribution-level plausibility is a fundamental challenge in conditional 3D medical image generation.
Approach: GDM reformulates deterministic image prediction as a multi-objective learning problem with one-step inference.
Mechanism: An attractive-repulsive drift minimizes the discrepancy between the generator pushforward and the target distribution.
Stability: Multi-level feature banks from a medical foundation encoder enable reliable affinity estimation and drift field computation on 3D volume data.
Optimization: A gradient reconciliation strategy in a shared output space balances distribution-level and fidelity-oriented objectives.
Evaluation: On MRI-to-CT synthesis and sparse-view CT reconstruction, GDM outperforms GAN, flow matching, and SDE-based generative models as well as supervised regression baselines.

---

*Auto-collected on 2026-04-23*