[论文] Variance Reduction for Expectations with Diffusion Teachers

论文概要

研究领域: CV 作者: Jesse Bettencourt, Xindi Wu, Matan Atzmon 发布时间: 2025-05-20 arXiv: 2505.15989

中文摘要

预训练扩散模型作为冻结教师为下游流程（如文生3D、单步蒸馏和数据归因）提供梯度。这些流程消费的教师梯度是关于噪声水平和高斯噪声样本的蒙特卡洛(MC)期望；其估计方差主导计算成本，因为每次采样都需要昂贵的上游工作（渲染、模拟、编码）。本文提出CARV，一个计算感知方差核算框架，推导出分层MC估计器：将昂贵的上游计算摊销到廉价的扩散噪声重采样上，并通过时间步重要性采样和分层逆CDF构造来锐化。在文生3D蒸馏和归因实验中，CARV在不改变目标的前提下实现2-3倍有效计算乘数（大部分来自摊销重用；约25%额外来自IS+分层）；在单步蒸馏中，相同技术将梯度方差降低一个数量级，但未改善下游FID，标志着MC方差不再是瓶颈的 regime。

原文摘要

Pretrained diffusion models serve as frozen teachers feeding downstream pipelines such as text-to-3D, single-step distillation, and data attribution. The teacher gradients these pipelines consume are Monte Carlo (MC) expectations over noise levels and Gaussian noise samples; their estimator variance dominates compute cost because each draw requires expensive upstream work (rendering, simulation, encoding). We introduce CARV, a compute-aware variance-accounting framework that motivates a hierarchical MC estimator: amortize the expensive upstream computation over cheap diffusion-noise resamples, sharpened by timestep importance sampling and a stratified-inverse-CDF construction. In our text-to-3D distillation and attribution experiments, CARV delivers 2-3x effective compute multipliers (most...

--- *自动采集于 2026-05-22*

#论文 #arXiv #CV #小凯