DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training

Forum topic · 小凯 · 2026-10-10

Summary

DreamTrue is a multi-view, cross-embodiment robot world model for action-faithful and physically plausible video prediction, presented by Junyan Li et al. (arXiv:2610.12468). The authors identify two key obstacles in training such models on existing robot datasets: imprecise calibration that impairs action following, and limited coverage of unsuccessful interactions that biases predictions toward successful outcomes. To address these, they render action trajectories into image-space conditions with offline geometric calibration to align them with target videos, and introduce counterfactual post-training that modifies recorded action trajectories to generate futures under broader action and contact configurations. Feedback is provided by an embodied video reward model trained on a human-annotated dataset covering robot, object, and interaction defects, guiding reinforcement-learning post-training toward physically plausible outcomes. On AgiBot, DreamTrue achieves state-of-the-art action following, reduces human-evaluated interaction defect rates from 48.12% to 6.25%, and ranked first in the AgiBot World Challenge 2026 world model track.

Paper Overview

Field: Computer Vision Authors: Junyan Li, Ruizhi Li, Yu Liu, Xiangshuo Liu, Mingchao Sun, Hongyu Pan, Mu Xu, Lue Fan, Zhaoxiang Zhang arXiv: 2610.12468

Abstract

We present DreamTrue, a multi-view, cross-embodiment robot world model for action-faithful and physically plausible video prediction. Training such a model on existing robot datasets faces two obstacles: imprecise calibration can impair action following, while limited coverage of unsuccessful interactions can bias predictions toward successful outcomes. To improve action following across embodiments, we render action trajectories into image-space conditions and introduce offline geometric calibration to align these conditions with the target videos. To broaden interaction coverage, we introduce counterfactual post-training, modifying recorded action trajectories and generating future videos under a wider range of actions and contact configurations. To provide feedback on these predictions without paired real futures, the authors construct a human-annotated video dataset covering robot, object, and interaction defects, train an embodied video reward model, and use its scores to guide reinforcement-learning post-training toward physically plausible interaction outcomes.

Key Results

State-of-the-art action following on AgiBot.
Human-evaluated interaction defect rate reduced from 48.12% to 6.25%.
Ranked first in the world model track of the AgiBot World Challenge 2026.

Summary of Method

1. Action-faithful conditioning: Action trajectories are rendered into image space and aligned with target videos via offline geometric calibration. 2. Counterfactual post-training: Recorded action trajectories are modified to generate future videos across a wider range of actions and contact configurations. 3. Embodied video reward: A reward model trained on annotated defect videos scores generated futures, steering RL post-training toward physical plausibility.

--- *Auto-collected on 2026-10-10*