SUN: Persistent Programs for Language-Grounded Control-to-Learning-to-Deployment in Robot Manipulation

Forum topic · 小凯 · 2026-09-02

Summary

Researchers Weiqi Wang, Zhi Li, and Yudong Lei introduce Semantically UNified (SUN) Programs, typed executables in which geometric and contact relations are defined once and compiled into aligned Model Predictive Control (MPC) costs, satisfaction predicates, RL rewards, transition guards, and diagnostics. This bridges a long-standing gap between model-based control and learned policies, where task semantics are typically discarded, forcing hand-crafted rewards and causing learned behavior to drift from control-verified results. The accompanying system, Kuafu, uses large vision-language systems to automatically synthesize SUN Programs from language instructions and scene semantics, screens feasibility via MPC, and preserves semantics when training stage-conditioned policies. Across nine tasks, Kuafu achieves an 82.03% macro success rate, outperforming sparse-reward (35.67%) and Stage-BC (24.75%) baselines. At 8192-way parallelism it generates successful trajectories 10.57x faster than human teleoperation. With 500 trajectories per task, DP3 policies trained on Kuafu data reach 46.0% success in simulation (vs. 22.4% for alternatives) and 34.7% on physical Franka and Kinova robots, without demonstrations or manual dense rewards.

Paper Overview

Field: Machine Learning Authors: Weiqi Wang, Zhi Li, Yudong Lei Published: 2025-09-01 arXiv: 2509.00140

Problem

Bridging model-based control and learned policies in long-horizon manipulation has harbored a silent disagreement: control executes specified objectives, learning amortizes that behavior into a reactive policy, yet existing protocols discard task semantics. This leaves rewards hand-crafted and behavior drifting from what control verified.

Method

The paper introduces Semantically UNified (SUN) Programs — typed executables where geometric and contact relations are defined once and compiled into:

Aligned Model Predictive Control (MPC) costs
Satisfaction predicates
RL rewards
Transition guards
Diagnostics

The accompanying system, Kuafu, is driven by large vision-language systems and:

1. Automatically synthesizes SUN Programs from language and scene semantics 2. Screens feasibility via MPC 3. Retains semantics while training stage-conditioned policies

Results

82.03% macro success rate across nine tasks, versus 35.67% for sparse-reward and 24.75% for Stage-BC baselines
At 8192-way parallelism, generates human teleoperation success trajectory time at 10.57x speed
With 500 trajectories per task, DP3 policies trained on Kuafu data reach 46.0% simulated success (alternatives: 22.4%) and 34.7% on physical Franka and Kinova robots

Conclusion

These results show that simulation-screened task semantics can effectively amortize control into robust policies without demonstrations or manually engineered dense rewards, unifying symbolic planning with data-driven execution.

---

*Originally collected on 2026-09-02*