A Robot Handing Over a Cabbage Stumps a Whole Row of Large Models
Here is a concrete scenario: on a factory line, a robot's left arm picks up a cabbage, passes it to its right arm, the right arm catches it, and places it at a designated spot. It sounds like the most basic coordinated motion. In an evaluation recently published by Alibaba DAMO Academy, this simple task knocked out a whole lineup of well-known models: π0.5 scored 0%, π0 scored 2.86%, and the best-performing traditional method, DP, managed only 17.14%.
DAMO's own model: 28.57%.
Yes, you read that right—the winner still falls short of thirty percent. Object handover remains unsolved by anyone, and it is precisely the task the paper most wanted to win.
What the work actually is
The paper is called RynnWorld-4D. Its technical report was released on July 8, 2026, with authors from Alibaba DAMO Academy, the Hong Kong Embodied AI Lab, CUHK, and Lakeside Lab. In one sentence: it aims to let robots not only perceive what the scene in front of them looks like, but also anticipate how its 3D structure will move after interaction.
Traditional world models learn to predict future video—showing you the next second's frames. But images are two-dimensional arrays of pixels; when the model draws a cup, the cup flickers in size within the frame and its shape subtly deforms. Flaws invisible to the human eye become disasters on a robot: the grasp position is computed wrong, the applied force is off. The paper sums up these defects in one phrase—2D models lack geometric grounding; object size fluctuates, shapes deform, and all of it turns into noise during policy learning.
DAMO's solution is to complete the prediction dimensions. Instead of predicting only frames, the model predicts three things simultaneously: color images (what it looks like), depth maps (how far away everything is), and optical flow (where everything is moving). All three predictions emerge from the same diffusion process, with the three branches mutually constraining each other—the predicted depth must be consistent with the predicted image, and the predicted motion must match the object displacements in the frame. If a chopstick shifts one centimeter in the image, the depth map must shift with it and the optical flow field must point the right way. Whenever one branch errs, the other two pull it back.
The training backbone is Alibaba's own video generation model, Wan2.2-TI2V-5B—a 30-layer Transformer with a three-branch structure and per-frame 3D rotary positional encodings. It was fed 254.4 million frames of video: egocentric human manipulation in kitchens, real manipulation trajectories from various robots, with depth and optical flow labels all produced as pseudo-labels by off-the-shelf models—not manually annotated. With over two hundred million frames, hand-labeling is out of the question.
Read the scorecard with level eyes
Six real-world bimanual manipulation tasks, each trajectory run on a physical robot (TIANJI M6 body, WUJI HAND dexterous hands): grasping an apple and a banana with both hands, pushing blocks with dual arms, handing over a cabbage, lifting a watermelon plush toy with both arms, putting on a lid, stacking bowls.
First place in all six. The delicate lid-placing task hits 65.71%, 8.57 percentage points above second place. Heavy lifting reaches 97.14%. Academically, this scorecard holds up.
But spread the whole table out and the story is more complete. On the simplest task, pushing blocks, π0.5 scores 100% versus DAMO's 97.14%—on basic tasks it is not dominantly ahead, and occasionally even loses. Its advantage concentrates where depth perception matters: stacking bowls, placing lids, handovers—exactly the scenarios where 2D models most easily make geometric errors. And with everyone wiped out on the handover task (best under 30%), the problem is clearly still too hard for all of today's models, including the winner.
The most informative row in the table uses the training backbone itself as the control. The researchers took the Wan2.2 video model (the version that predicts only frames, without depth and flow), attached the same policy interface, fed it the same robot data, and ran the comparison again: averaged across the six tasks, the full version beats the frame-only version by about nine and a half percentage points. This is the cleanest evidence in the entire table—the improvement comes not from a bigger model or more data, but precisely from the single act of "also predicting depth and motion."
Where the nine hertz comes from
The marketing says 9 Hz high-frequency closed-loop control. That number needs unpacking—the paper unpacks it itself.
Each full future prediction takes 1.1 seconds—on its own, a planning frequency under one hertz, two orders of magnitude below traditional industrial control's five hundred hertz. The 9 Hz works like this: the model predicts a string of ten actions in one shot; while the robot executes those ten actions, the GPU computes the next string in parallel. The execution layer reads actions from a lookup at 50 Hz while the prediction layer thinks slowly in the background. Spread out, the action string refreshes every 1.1 seconds, each string lasts 1.1 seconds, giving an effective control frequency of about nine hertz.
So strictly speaking, this "eye of time" blinks only 0.9 times per second, with action strings filling the gaps between blinks. The paper's limitations section states plainly that this latency remains a bottleneck for ultra-high-frequency control. Such candor deserves to be in the body text, not a footnote.
Bullets fired three months ago
Finally, the timeline. This work is not from last week: the technical report went online July 8, with code and model weights open-sourced on July 7; the repository has sat at 95 stars for three months. The paper received a v2 update on September 29. It recently re-entered discussion thanks to the experimental details completed in v2, and a question that is heating up: should world models carry geometry?
The world-model community is currently split into two camps. One bets on pure video: treating the world as a sequence of frames, Sora-style, gambling that "predicting pixels is enough." The other believes it is not: images can lie; depth and motion cannot. This work hands the second camp its most complete evidence so far—on the same backbone, the geometry-equipped version stably beats the frame-only version on a physical robot, with no exceptions across six tasks.
DAMO names this route "from 2D pixel prediction to physically grounded 4D scene evolution." The name is long, but its meaning can be finished with that one cabbage: knowing what the cabbage looks like is not enough—you must know how far it is from the left gripper, when to release, and where the right gripper should close. None of these are things pixels can directly tell you.
As for when robot handover success rates can climb from thirty percent to ninety—the 0% and 2.86% entries in that table will first answer a different question: this problem is even harder than it looks.
---
*Sources: RynnWorld-4D technical report (arXiv 2607.06559, v2 2026-09-29; Alibaba DAMO Academy × Hong Kong Embodied AI Lab × CUHK × Lakeside Lab); code and weights open-sourced 2026-07-07 (alibaba-damo-academy/RynnWorld-4D, 95 stars); main success-rate table Table 5 (measured on TIANJI M6 + WUJI HAND); latency table Table 1 (RTX 5090 FP8).*