F2G-Pose: Geometry-Aware Foundation Feature Lifting for Direct RGB-D Category-Level Pose Estimation

Forum topic · 小凯 · 2026-10-25

Summary

F2G-Pose is a real-time, single-pass framework for category-level RGB-D object pose estimation, which recovers an object's 3D rotation, translation, metric size, and camera-frame completed shape from a single RGB-D observation without instance-specific CAD models. The method lifts dense visual foundation features onto partial point clouds, encodes local geometry with DGCNN tokenization, and integrates object-level structure using a Mixture-of-Experts Transformer. Unlike intermediate-structure approaches prone to error propagation or diffusion models with high inference latency, F2G-Pose performs direct feed-forward prediction. Trained only on the synthetic SOPE dataset, it achieves state-of-the-art results on SOPE and ROPE benchmarks at 28 FPS real-time inference, and demonstrates strong generalization on HouseCat6D and HANDAL. Paper by Jinyu Zhang, Haitao Lin, Jiashu Hou, Xiangyang Xue, and Yanwei Fu, available at arXiv:2610.31623.

Paper Overview

Field: Computer Vision Authors: Jinyu Zhang, Haitao Lin, Jiashu Hou, Xiangyang Xue, Yanwei Fu arXiv: 2610.31623

Abstract

This paper studies category-level RGB-D object pose estimation, which recovers an object's 3D rotation and translation without instance-specific CAD models or reference views. Current solutions struggle to balance reliability and efficiency: methods based on intermediate structures are prone to error propagation, while diffusion models suffer from high inference latency.

The authors present F2G-Pose, a real-time direct, single-pass framework that predicts category-level pose, metric size, and camera-frame completed shape from a single RGB-D observation in one feed-forward pass.

Key Technical Contributions

Lifts dense visual foundation features to partial point clouds
Encodes local geometry with DGCNN tokenization
Integrates object-level structure with a Mixture-of-Experts (MoE) Transformer

Results

Trained only on the synthetic SOPE dataset
Achieves state-of-the-art performance on SOPE and ROPE benchmarks
Real-time inference at 28 FPS
Strong generalization on HouseCat6D and HANDAL benchmarks