StreamPI Adds the Time Dimension to VLA Robots for Just 9.2 ms

Forum topic · 小凯 · 2026-08-30

Summary

Most vision-language-action (VLA) models, including Physical Intelligence's π0.5, still operate on a single-frame paradigm: see one image, act once. StreamPI, an arXiv paper (2608.26067) from a HKU-led team, upgrades robots from photos to video by packing each timestep into a visual observation plus language instruction unit—bidirectional attention within units, causal attention across units, instructions re-anchored every step as "semantic anchors," and past frames reused via a rolling KV cache with zero added parameters. Trained with random temporal gaps, it tolerates dropped frames and keeps 97.1 on LIBERO even with a single frame at inference. Benchmarks: LIBERO 98.3 vs π0.5's 96.9 (LIBERO-Long 95.0 vs 92.4); CALVIN average chain length 4.547 vs π0.5's 4.313 and MemoryVLA's 4.090. On real-robot tasks, Shell Game success jumped from 46.7% to 80%, rolling-object grasping from 26.7% to 63.3%. Cost: inference latency rises only 9.2 ms (94.4 to 103.6 ms on RTX 4090). Code is open-sourced at hku-sail/StreamPI, built on PI's openpi.

Ask yourself: staring at one kitchen photo, can you tell whether the water in the kettle is sloshing right now, or whether the lid is being lifted versus just placed? A photo contains no "currently," only "is." Yet mainstream VLA models—including Physical Intelligence's flagship π0.5—run on exactly this single-frame paradigm: look, act, look again.

StreamPI (arXiv:2608.26067, posted August 26) upgrades robots from photos to video.

Method

Each timestep is packed as a "visual observation + language instruction" sequence unit: bidirectional attention inside units for image-text fusion, causal attention across units so the model only sees the past.
Instructions are re-bound at every step as "semantic anchors," so the robot doesn't forget its goal mid-task.
Historical frames are not recomputed—they roll through a KV cache. Crucially, zero new parameters: no video encoder, no memory module, just surgery on π0.5's attention structure.
Training uses random temporal gaps and randomly dropped early frames, making the model robust to camera stutter and frame loss. It trains on T=5 frames but keeps a 97.1 LIBERO average with only 1 frame at inference.

Results

LIBERO: π0.5 averages 96.9; StreamPI (5 frames) hits 98.3. On memory-heavy LIBERO-Long: 92.4 → 95.0.
CALVIN long-horizon chains: average task length 4.547, beating π0.5's 4.313 and the memory-specialized MemoryVLA's 4.090. At chain step five, MemoryVLA drops to 69.4% while StreamPI holds 85.0%.
Real-robot tasks (15–30 evaluations each): Shell Game 46.7% → 80.0%—single-frame models don't know the cups were just swapped; rolling-object grasping 26.7% → 63.3%; narrow-mouth pen insertion 40% → 66.7%; cup-into-sleeve 60% → 92%.
Cost: inference latency on an RTX 4090 goes from 94.4 ms to 103.6 ms—only 9.2 ms for five frames, thanks to cache reuse.

Caveats

1. Flashy phrases like "continuous physical intelligence" come from the press release, not the paper, which calls itself streaming multimodal temporal modeling. 2. The press release's "96.4% → 97.5%" baseline comes from an ablation table, not π0.5's actual 96.9—don't underestimate the starting point. 3. Real-robot tasks have only 15–30 evaluations with no error bars; a single run is worth 3–7 points, so the +33.3 Shell Game jump may not reproduce so cleanly.

Attribution notes

"Daxiao Robotics" in Chinese coverage is ACE ROBOTICS, a Shanghai company chaired by SenseTime co-founder Wang Xiaogang, with Tao Dacheng as chief scientist; Ant Group led its angel round (an investor, not an owner). Ant's own embodied company is Robbyant, which builds LingBot-VLA—a different company and model.
StreamPI's technical base is Physical Intelligence's open-source openpi.
Eight of ten authors are from HKU (corresponding author Hengshuang Zhao, SAIL lab). Code: hku-sail/StreamPI; project page: happinesslz.github.io/projects/StreamPI.

While embodied AI has raced on parameters, data, and control frequency, the time dimension stayed empty. StreamPI fills it for 9.2 ms.

References: arXiv:2608.26067 | GitHub | Project page | leiphone | qbitai