Paper Overview
Field: Machine Learning / Robotics Authors: Shivin Dass, Kevin Nguyen, Rohan Yelandur, Junhong Xu, Roberto Martín-Martín Published: 2026-11-03 arXiv: 2611.05368
Abstract
Imitation learning policies in robotics are trained on datasets collected through costly real-world teleoperation. Often, the data-collection process is unguided, relying on the teleoperator's intuition, resulting in data that contributes little to overall policy performance. Recent work has shown that curating such unguided datasets can substantially improve policy performance, but these approaches operate by filtering existing datasets after collection and therefore do not prevent the acquisition of uninformative data.
In this work, the authors propose GUILD, a novel Guided Data Collection (GDC) method that actively reasons about which data should be collected to maximize downstream policy performance. The framework is an iterative loop alternating between:
1. Task proposal and data collection — leveraging semantic priors from vision-language models to identify relevant environments, object configurations, and goals for data collection. 2. Data evaluation — since semantic relevance does not imply policy improvement, predictive data attribution is used to approximate how the collected trajectories influence policy performance on a target deployment setting. This signal is fed back to guide subsequent data collection, closing the loop.
Results
Evaluated in both simulation and real robot settings, guided data collection substantially improves sample efficiency and deployment performance compared to passive data collection strategies, multiple baselines, and even experienced roboticists.
More information: https://robin-lab.cs.utexas.edu/guild2026/