English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning

Forum topic · 小凯 · 2026-08-13

Summary

Surgical WAM is a unified generative world-action model, built on Cosmos Policy, that jointly predicts future endoscopic observations and executable surgical robot action chunks. It addresses the bottleneck of scarce action-labeled surgical demonstrations by first pretraining on abundant action-free endoscopic video to learn surgical visual dynamics, then fine-tuning on a fixed action-labeled demonstration budget. At deployment, it operates as a closed-loop receding-horizon controller, executing short prefixes of predicted action chunks and replanning from resulting observations. Evaluated on four simulated surgical manipulation task suites, video pretraining raises average success rate from 63.5% to 77.8%, with a 20-percentage-point absolute gain on PegTransfer and the largest improvements on contact-rich and bimanual tasks. Results show action-free video provides transferable visual dynamics priors, making data-efficient pretraining a practical path for scaling surgical robot learning.

Overview

Field: Computer Vision Authors: Wenrui Bao, Tianyun Jiang, Zhiben Chen, Ser-Nam Lim, Peter D. Peng, Yuzhang Shang Published: 2026-08-11 arXiv: 2608.11204

Motivation

Learning reliable surgical manipulation policies is bottlenecked by the scarcity of action-labeled demonstrations: teleoperated surgical robot (e.g., dVRK) trajectories with synchronized kinematics are costly to collect, while surgical tasks demand precise contact handling, long-horizon reasoning, and bimanual coordination.

Endoscopic video is comparatively inexpensive and abundant relative to synchronized video–kinematics trajectories, and a natural way to exploit it is to learn world models of surgical scenes. However, existing surgical world models use video primarily for simulation or policy evaluation, and rarely translate the learned dynamics into closed-loop control.

This gap raises the paper's central question: under a fixed budget of action-labeled demonstrations, does action-free video pretraining improve closed-loop surgical manipulation?

Method

  • Surgical WAM (Surgical World-Action Model) is a unified generative model based on Cosmos Policy that jointly predicts future endoscopic observations and executable surgical robot action chunks.
  • It first learns surgical visual dynamics from action-free video, then fine-tunes on a fixed action-labeled demonstration budget.
  • At deployment, it acts as a closed-loop receding-horizon controller: it executes the short prefix of each predicted action chunk and replans from the resulting observations.
  • Results

  • On four simulated surgical manipulation task suites, video pretraining improves average success rate from 63.5% to 77.8%.
  • PegTransfer gains an absolute 20 percentage points.
  • The largest improvements appear on contact-rich and bimanual tasks.

Takeaway

Action-free video provides transferable visual dynamics priors for surgical robot control learning under limited action supervision, making data-efficient video pretraining a practical path for scaling surgical robot learning.

--- *Auto-collected on 2026-08-13*

Tags

#surgical-robotics#world-model#video-pretraining#imitation-learning#computer-vision#manipulation#arxiv#cosmos-policy

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633399