GDM: Generative Drifting for Conditional 3D Medical Image Generation

Forum topic · 小凯 · 2026-04-23

Summary

Researchers propose GDM, a generative drifting framework for conditional 3D medical image generation. The method reformulates deterministic medical image prediction as a multi-objective learning problem that jointly optimizes distribution-level plausibility and patient-specific fidelity while retaining one-step inference efficiency. GDM extends drifting to 3D medical imaging via an attractive-repulsive drift that minimizes the discrepancy between the generator pushforward and the target distribution. For stable drifting-based learning on volumetric data, it builds multi-level feature banks from a medical foundation encoder, supporting reliable affinity estimation and drift-field computation across complementary global, local, and spatial representations. A gradient harmonization strategy in a shared output space balances the competing distribution-level and fidelity-oriented objectives. Evaluated on MRI-to-CT synthesis and sparse-view CT reconstruction, GDM consistently outperforms GAN-based, flow-matching, and SDE-based generative baselines as well as supervised regression methods, improving the trade-off among anatomical fidelity, quantitative reliability, perceptual realism, and inference efficiency. Paper: arXiv 2604.19736.

Paper Overview

Field: Computer Vision Authors: Zirong Li, Siyuan Mei, Weiwen Wu, Andreas Maier, Lina Gölz, Yan Xia Published: 2026-04-21 arXiv: 2604.19736

Abstract

Conditional medical image generation plays an important role in many clinically relevant imaging tasks. However, existing methods still face a fundamental challenge in balancing inference efficiency, patient-specific fidelity, and distribution-level plausibility, particularly in high-dimensional 3D medical imaging. In this work, the authors propose GDM, a generative drifting framework that reformulates deterministic medical image prediction as a multi-objective learning problem to jointly promote distribution-level plausibility and patient-specific fidelity while retaining one-step inference.

Key Points

Generative drifting: GDM extends drifting to 3D medical imaging through an attractive-repulsive drift that minimizes the discrepancy between the generator pushforward and the target distribution.
Stable drifting on volumetric data: To enable stable drifting-based learning on 3D volumes, GDM constructs multi-level feature banks from a medical foundation encoder, supporting reliable affinity estimation and drift-field computation across complementary global, local, and spatial representations.
Gradient harmonization: A gradient harmonization strategy in a shared output space improves the optimization balance between competing distribution-level and fidelity-oriented objectives.
Tasks evaluated: MRI-to-CT synthesis and sparse-view CT reconstruction.
Results: GDM consistently outperforms a broad range of baselines, including GAN-based, flow-matching, and SDE-based generative models as well as supervised regression methods, while improving the balance among anatomical fidelity, quantitative reliability, perceptual realism, and inference efficiency.

Conclusion

GDM offers a practical and effective framework for conditional 3D medical image generation, combining single-step inference with improved plausibility and patient-specific fidelity.

---

*Auto-collected on 2026-04-23*