[论文] TTB: Test-time Baking for Efficient Rendering of Decoder-only View Syn...

论文概要

研究领域: CV 作者: Chin-Yang Lin, Ryo Hachiuma, Jaesung Choe, Min-Hung Chen, Yu-Chiang Frank Wang, Yen-Yu Lin, Wei-Chen Chiu, Yu-Lun Liu, Cheng Sun 发布时间: 2026-11-12 arXiv: 2611.17331

中文摘要

带有 KV 缓存的仅解码器大型视图合成模型（LVSM）最近通过神经网络直接回归新视图、无需重建 3D 几何，达到了最先进的质量；而测试时训练（TTT）层虽然节省计算，却牺牲了质量。我们提出测试时烘焙（TTB），通过学习将全局场景信息从跨视图注意力"烘焙"进一个轻量级 MLP，实现快速且高保真的新视图解码。第一个关键点是重新利用 TTT 的快速权重 MLP，从其输入输出对中显式学习跨视图注意力映射，而非完全替换注意力——在保持与 TTT 相同渲染效率的同时，质量显著提升。第二个关键点解决上下文预填充与新视图渲染之间的模态错位：我们将上下文 token 解耦为仅含相机信息的渲染 token（作为 TTB 输入）和含图像内容的真值 token（作为 TTB 映射目标），确保烘焙出的 MLP 无需承担跨模态泛化的负担。我们的方法在多个基准上实现了最高 23 倍的渲染加速，质量与最先进方法相当。

原文摘要

Decoder-only Large View Synthesis Models (LVSMs) with KV-cache have recently achieved state-of-the-art quality by regressing novel views from neural networks without reconstructing 3D geometry, while Test-Time Training (TTT) layers save computation but sacrifice quality. We propose Test-Time Baking (TTB), a strategy that learns to bake global scene information from cross-view attention into a lightweight MLP, enabling fast and high-fidelity novel-view decoding. The first key is to repurpose the fast-weight MLP of TTT to learn the cross-view attention mapping explicitly from its input-output pairs, rather than replacing attention entirely, achieving the same rendering efficiency as TTT while delivering substantially higher quality. The second key addresses a modality misalignment between co...

#论文 #arXiv #CV #小凯