SG-DOR: Scene Graph and Direction-Aware Occlusion Reasoning for Pepper-Picking Robots

Forum topic · 小凯 · 2026-03-09

Summary

SG-DOR is a research framework that reframes robotic pepper harvesting from a purely geometric problem into a relational reasoning task. Traditional harvest pipelines (detect, localize, grasp) fail when leaves occlude fruit, and famous systems like SWEEPER achieved only ~58% success in 2018. SG-DOR builds a scene graph whose nodes represent fruit, leaves, stems, and branches, with edges encoding physical connections and spatial occlusion. A direction-conditioned occlusion reasoning module, implemented with per-fruit leaf-set attention and union-level aggregation in a graph neural network, predicts which leaves block a candidate picking direction. Evaluated on synthetic pepper datasets, it reports 0.73 occlusion-prediction F1, 0.85 NDCG@3, and 0.83 edge F1 for connection reasoning. Using synthetic data avoids costly real-world annotation, and the authors estimate potential harvesting success gains from ~60% to ~80%. Limitations include untested sim-to-real transfer, static scenes without contact deformation, and single-plant scenarios.

The Core Problem: The "Last Mile" of Agricultural Robotics

Traditional robotic harvesting systems follow a "detect → localize → grasp" pipeline, but face several challenges:

Occlusion: fruit is frequently hidden behind leaves
Unclear physical relations: the robot cannot tell which leaves are "in the way" versus "physically connected"
Direction sensitivity: approaching from different angles gets blocked by different leaves
Dynamic environments: plants are not rigid bodies and deform on contact

The SWEEPER robot (the best-known sweet pepper harvester) achieved only a 58% success rate in 2018.

Key Innovations of SG-DOR

1. Scene Graph

Nodes: fruit, leaves, stems, branches
Edges: physical connections and spatial occlusion relations

2. Direction-Conditioned Occlusion Reasoning

Technical Architecture

Input: instance-segmented organ point clouds
Core modules:
Scene graph construction (nodes + edges)
Occlusion ranking task
Direction-aware graph neural network (per-fruit leaf-set attention + union-level aggregation)
Output: structured relational signals for downstream harvesting motion planning

Experimental Results

Tested on a synthetic pepper dataset:

| Metric | Result | |--------|--------| | Occlusion prediction F1 | 0.73 | | NDCG@3 | 0.85 | | Connection reasoning Edge F1 | 0.83 |

Significance

1. From "seeing" to "understanding": addresses the question of what the relations actually are 2. Data efficiency: synthetic datasets avoid expensive real-world annotation 3. Direct application value: could raise harvesting success from ~60% to ~80%

Limitations

Synthetic-to-real performance remains unverified
Static scenes only; contact-induced deformation is not modeled
Single-plant setting; multi-plant interactions are not covered

One-Sentence Takeaway

SG-DOR turns pepper picking from a geometric problem into a relational reasoning problem—instead of merely "seeing" fruit and leaves, the robot "understands" their spatial relations and physical connections.