SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators

Paper: SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators Authors: Yuncong Yang, Zhengtao Han, Furkan Ozyurt, et al. arXiv: 2609.09155 Institutions: UMass Amherst, UC Berkeley, NYU, Harvard

The Problem: The Tower of Babel for World Models

World models predict the next visual frame given the current frame and an action, allowing robots to rehearse action sequences in imagination before acting. But actions are not a universal language in pixel space:

• Camera position changes: the same "move forward 10 cm" produces different pixel displacements
• Robot arm swapped: identical control signals yield different trajectories
• Environment changes: the same action looks different against different backgrounds
• Robot base moved: the relation between global coordinates and the local view shifts

The traditional fix—retraining for every new setup—is impractical in real deployments.

Core Idea: Calibration as In-Context Learning

SyncWorld's insight is to stop trying to learn one universal action-to-vision mapping, and instead learn how to infer the current environment's mapping rule from a brief calibration.

In a new environment, the robot performs a short "calibration episode": a series of simple exploratory actions demonstrating all six degrees of freedom (DoF), while recording the corresponding visual changes. These paired action-visual examples serve as in-context examples—much like few-shot prompting in language models—letting the model resolve the setup-dependent Action-Visual Mapping without any gradient updates.

This yields:

• Zero-shot generalization to new environments via a brief calibration
• Cross-environment transfer with a single model
• Cross-embodiment generalization across different robot arms

Training: Calibration Distillation

SyncWorld is trained via a strategy called Calibration Distillation:

1. Multi-source data mixture: training data spans varied camera viewpoints, arms, and backgrounds 2. Injected calibration contexts: calibration episodes are systematically provided as context during training 3. Learning objective: the model learns to use calibration information to interpret setup-specific mappings, mitigating conflicting supervision (the same action values corresponding to incompatible visual changes across setups)

Notably, SyncWorld also learns history-aware implicit calibration: when no explicit calibration episode is available, it can infer the environment's mapping from prior interaction history—valuable in real deployments where frequent recalibration is impractical.

Experimental Results

Evaluated on challenging benchmarks involving unseen camera viewpoints, unseen embodiments, and unseen backgrounds:

• SyncWorld accurately simulates action outcomes in previously unseen environments, matching or exceeding prior state-of-the-art methods
• Cross-embodiment generalization is a particular strength
• Calibration quality matters: a complete demonstration of all 6 DoFs works best, but even a few-second calibration enables effective adaptation

Test-Time Policy Improvement

SyncWorld also supports test-time policy improvement: the robot imagines multiple candidate action sequences in the world model, evaluates their predicted outcomes, and executes the best one—no additional training required. The authors demonstrate significant policy gains on real robots in new environments.

Why It Matters

Real-world world models must adapt to constantly changing conditions: lighting shifts, bumped cameras, product variations, post-maintenance recalibration. SyncWorld's zero-shot adaptation—seconds of automatic calibration instead of retraining—is a key step from lab prototypes to industrial deployment, mirroring how humans rapidly adapt to new glasses, new cars, or new rooms through quick contextual recalibration.

Promising future directions include online continual calibration, multimodal calibration (force/touch), social calibration in multi-robot systems, and meta-learning optimal calibration action sequences.

References

• Yang, Y., Han, Z., Ozyurt, F., et al. (2026). *SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators*. arXiv:2609.09155.
• Hafner, D., et al. (2020). Dream to Control: Learning Behaviors by Latent Imagination. *ICML*.
• Ebert, F., et al. (2018). Visual Foresight: Model-Based Deep RL for Vision-Based Robotic Control. *CoRL*.
• Ye, S., et al. (2026). World Action Models are Zero-Shot Policies. arXiv:2602.15922.