EyeRobot 2.0: Active Gaze for Precise Manipulation Without a Wrist Camera

Forum topic · 小凯 · 2026-10-06

Summary

EyeRobot 2.0 is a robotics framework from researchers including Kush Hari, Justin Kerr, Ken Goldberg, and Angjoo Kanazawa (arXiv:2610.03710) that enables fine-grained bimanual manipulation using only a single stereo camera. Inspired by human vision, the robot physically attends to a 3D fixation point by swiveling two eye viewpoints onto it, and processes resulting images foveally—allocating more visual tokens to image centers to focus computation on task-relevant features. This Active Visual Fixation (AVF) is trained hierarchically with reinforcement learning on real-world data: a low-level gaze servoing policy conditioned on a goal object, plus a target selector co-trained with a BC gripper policy that emits fixation goals based on task progress. Gaze information is also used to normalize gripper poses into gaze-relative SE(3) coordinates. Across 7 real-world tasks and 6 simulated tasks (1000+ physical trials, 1800+ simulated trials), removing the wrist camera hurt standard policies badly: real-world success dropped from 52% to 27% with passive stereo alone. EyeRobot 2.0 closed the gap, outperforming passive stereo by 40% in real settings and 20% in simulation, matching ego+wrist policies when the wrist view is clear, and more than doubling success when the wrist camera is occluded.

EyeRobot 2.0: Active Gaze for Precise Manipulation Without a Wrist Camera

Field: Machine Learning Authors: Kush Hari, Justin Kerr, Nidhya Shivakumar, Samarth Mahapatra, Carmelo Sferrazza, Jiahui Lei, Jitendra Malik, C. Karen Liu, Ken Goldberg, Angjoo Kanazawa Published: 2026-10-02 arXiv: 2610.03710

Summary

Inspired by human vision, the authors introduce a framework using active gaze to enable fine-grained bimanual manipulation with only a single stereo camera. EyeRobot 2.0 physically attends to a 3D fixation point in the scene by swiveling two eye viewpoints to center their gaze on it. The resulting images are processed foveally by allocating more visual tokens to the image centers, focusing computation on task-relevant features.

Such Active Visual Fixation (AVF) requires carefully coordinated gaze during task execution, accomplished hierarchically: first a low-level gaze servoing policy is trained, conditioned on a goal object, then a target selector is trained to emit fixation goals based on task progress. Both modules are trained with RL on real-world data—the former with dense geometric rewards, the latter co-trained with a BC gripper policy, allowing it to discover human-like gaze sequences during task execution.

EyeRobot 2.0 further leverages gaze information to normalize gripper poses into a gaze-relative SE(3) coordinate frame, compressing the scale of action distributions the policy must learn.

Evaluation

The authors collected teleoperation data for 7 real-world tasks and 6 simulated tasks, running over 1,000 physical experiments and 1,800 simulated trials.

Key results:

Removing the wrist camera is costly for standard policies: with only passive stereo vision, real-world success drops from 52% to 27%.
EyeRobot 2.0 closes this gap with stereo cameras only: +40% over passive stereo in the real world, +20% in simulation.
When the wrist view is unobstructed, it matches the ego+wrist policy (69% vs 64%).
When grasped objects occlude the wrist camera, success more than doubles (48% vs 22%).

Source

Paper: https://arxiv.org/abs/2610.03710

*Auto-collected on 2026-10-06.*

EyeRobot 2.0: Active Gaze for Precise Manipulation Without a Wrist Camera

Summary

EyeRobot 2.0: Active Gaze for Precise Manipulation Without a Wrist Camera

Summary

Evaluation

Source

Tags