Paper Overview
Research Area: Computer Vision Authors: Yiwen Chen, Qiqi Hou, Jisoo Jeong, Yinhao Zhu, Huaizu Jiang, Amir Said Published: 2026-11-04 arXiv: 2611.06901
Summary
JEPA-style world models learn action-conditioned latent dynamics without reconstructing observations, but distributional regularization alone does not ensure that their representations preserve information needed for control. The authors identify static collapse, a failure mode in which the encoder preserves episode-specific static content while suppressing manipulation-relevant state changes.
Because static content is temporally predictable yet diverse across episodes, this failure can persist despite regularizers such as SIGReg. A controlled study on OGBench-Cube shows that introducing randomized backgrounds substantially degrades planning performance while leaving the underlying manipulation task unchanged.
Proposed Method
To address this problem, the authors propose motion-guided contrastive regularization:
- Predicted latent transitions are compared against transitions of candidate observations from the same episode.
- This reduces the usefulness of static background identity for distinguishing action outcomes.
- Optical-flow similarity provides a soft contrastive target, so motion-similar candidates receive similar supervision.
- Combined with latent prediction and SIGReg, the objective encourages representations that preserve within-episode state distinctions and similarity across related transitions.
Results
| Benchmark | Baseline (LeWM) | Proposed Method | |---|---|---| | OGBench-Cube (randomized backgrounds) | 52% | 78% | | LIBERO-Object | 14% | 54% |
Performance remains comparable on four other visual control benchmarks.
Original Abstract (excerpt)
> JEPA-style world models learn action-conditioned latent dynamics without reconstructing observations, but distributional regularization alone does not ensure that their representations preserve information needed for control. We identify static collapse, a failure mode in which the encoder preserves episode-specific static content while suppressing manipulation-relevant state changes. Because static content is temporally predictable yet diverse across episodes, this failure can persist despite regularizers such as SIGReg. A controlled study on OGBench-Cube shows that introducing randomized backgrounds substantially degrades planning performance while leaving the underlying manipulation task unchanged. To address this problem, we propose motion-guided contrastive regularization. Our approac...
Read the full paper: arXiv:2611.06901
--- *Auto-collected on 2026-11-06*