[论文] T2-z: Test-Time Optimization of Environment Latents for Robotic World ...

论文概要

研究领域: CV 作者: Chenyuan Zhou, Yining Hong, Huang Huang, Yunfan Jiang, Wenlong Huang, David Held, Vitor Campagnolo Guizilini, Yue Wang, Li Fei-Fei, Jiajun Wu, Leonidas Guibas, Yejin Choi 发布时间: 2026-10-15 arXiv: 2610.22364

中文摘要

预测世界对动作的反应需要超越可见信息的能力：机器人必须考虑既不可直接观察也未明确指定的环境因素，从摩擦和质量等物理属性到人为定义的环境规则。我们引入 T2-Z 进行环境潜变量的测试时优化，用可学习的环境潜变量增强机器人世界模型以条件化世界预测。这个紧凑的潜变量捕捉底层环境变异，无需监督环境因素是什么或取什么值。训练期间，T2-Z 在世界模型优化和环境潜变量优化两个阶段之间交替。测试时，世界模型保持冻结，新初始化的环境潜变量从一个或多个测试时交互中优化，使模型适应未见过的环境配置。在 7 个模拟环境和 4 个真实机器人任务中，T2-Z 将物体预测误差从 15.20 降至 7.00，动作成功率从 33.73% 提升至 72.96%。消融实验表明，T2-Z 可以仅从单个测试时交互轨迹中高效推断环境潜变量。

原文摘要

Predicting how the world responds to an action requires more than what can be seen: robots must account for environmental factors that are neither directly observable nor explicitly specified, from physical properties such as friction and mass to human-defined environmental rules. We introduce T2-Z for test-time optimization of environment latents, which augments a robotic world model with a learnable environment latent to condition world predictions. This compact latent captures underlying environmental variation without supervision of what environmental factors exist or what values they take. During training, T2-Z alternates between two stages: a world-model optimization stage that fixes the latents and learns general action-conditioned dynamics across environments; and an environment-la...

#论文 #arXiv #CV #小凯