InstantFusion: A Shared Latent Interface Connecting Heterogeneous Image Generators

Forum topic · 小凯 · 2026-10-13

Summary

InstantFusion (arXiv:2610.13944) is a lightweight shared latent interface that connects the denoising trajectories of heterogeneous diffusion and flow-based image generators. The authors observe that independently trained generators exhibit similar latent spatial organization, and that directly handing off intermediate states between unaligned models preserves coarse visual content. Building on this, InstantFusion introduces model-specific Latent AutoEncoders (LAEs) that project each generator's intermediate states into a shared latent space and decode them into a target generator's latent space. Training is constrained by two complementary objectives: aligning intermediate states of the same image at matched noise levels, and synchronizing their corresponding local denoising velocities. Once trained, denoising can switch seamlessly between frozen generators without per-pair mapping training. The method demonstrates competitive results in acceleration, multi-preference composition, on-policy cross-architecture distillation, and cross-model composition.

Paper Overview

Field: Computer Vision Authors: Xuanlang Dai, Jiazi Bu, Yujie Zhou, Yuhong Liu, Beichen Zhang, Penghui Yang, Ziyu Liu, Yuhang Zang Published: 2026-10-09 arXiv: 2610.13944

Abstract (English)

Diffusion/flow models have emerged as the de facto paradigms for image generation, giving rise to a diverse family of modern generators with increasingly heterogeneous architectures and representations. Despite their remarkable generation capabilities, the cross-model alignment of latent representations and the consistency of their underlying denoising dynamics remain underexplored.

In this work, we uncover an intriguing cross-model compatibility: independently trained generators exhibit similar latent spatial organization, while directly handing off intermediate states between unaligned models can still preserve coarse visual content. Motivated by these observations, we propose InstantFusion, a lightweight shared latent interface that connects the denoising trajectories of heterogeneous image generators.

Method Highlights

Latent AutoEncoders (LAE): model-specific encoders project intermediate states of different generators into a shared latent space, then decode into the target generator's latent space.
Dual training objectives:
Align intermediate states of the same image at matched noise levels.
Synchronize the corresponding local denoising velocities.
Training-free switching: after training, denoising can seamlessly hand off between frozen generators — no dedicated mapping needs to be trained for each pair of models.

Results

InstantFusion shows competitive performance across multiple scenarios:

Generation acceleration
Multi-preference composition
On-policy distillation across architectures
Cross-model composition

--- *Auto-collected on 2026-10-13*