[论文] GROW²: Grounding Which and Where for Robot Tool Use

论文概要

研究领域: 机器人
作者: Yuhong Deng, Yuyao Liu, David Hsu
发布时间: 2026-07-01
arXiv: 2507.00006

中文摘要

如果没有刀，机器人能用盘子切蛋糕吗？工具使用极大地扩展了机器人的能力，但为了创造性地使用工具超越其预期功能，机器人面临开放世界可供性grounding的挑战：选择一个开放类别物体作为工具，并定位其特定的作用区域。为此，我们引入GROW²（Grounding Which and Where），它利用物体部件作为自然抽象，将grounding过程分层拆分为语义和几何级别，从而规避了对数据密集型端到端训练的需求。在语义上，GROW²利用视觉语言模型（VLM）的常识推理来解析自然语言任务指令，选择合适的物体作为工具，并识别工具和目标物体上与任务相关的部件。在几何上，视觉基础模型然后将选定的部件从单个RGB-D图像grounding到精确的3D区域。在既定基准上的实验表明，GROW²在可供性预测基准上优于最先进的基线。此外，它在开放类别物体上实现了零样本泛化，并在模拟和真实世界机器人工具使用实验中都优于基线。

原文摘要

Can the robot use a plate to cut a cake if no knife is available? Tool use greatly expands robot capabilities, but to use tools creatively beyond their intended functions, the robot faces the challenge of open-world affordance grounding: select an open-category object to act as a tool and localize its specific region of action. To this end, we introduce GROW² (GROunding Which and Where), which leverages object parts as a natural abstraction to split the grounding process hierarchically into semantic and geometric levels, thus bypassing the need for data-heavy, end-to-end training. Semantically, GROW² harnesses the commonsense reasoning of Vision-Language Models (VLMs) to parse a natural-language task instruction, select a suitable object as the tool, and identify task-relevant parts on the...

自动采集于 2026-07-01

#论文 #arXiv #机器人 #小凯