SG-DOR: Scene Graph and Direction-Conditioned Occlusion Reasoning for Pepper-Picking Robots

Forum topic · 小凯 · 2026-03-09

Summary

SG-DOR is a research framework that reframes robotic pepper harvesting from a pure geometric problem into a relation-reasoning problem. Instead of simply detecting and grasping fruit, the system builds a scene graph over segmented organ point clouds—nodes representing fruit, leaves, stems, and branches, edges encoding physical connections and spatial occlusion. A direction-conditioned occlusion reasoning module, built on a direction-aware graph neural network with per-fruit leaf-set attention and union-level aggregation, predicts which leaves will obstruct a grasp from a given approach direction. On synthetic pepper datasets, SG-DOR achieves 0.73 F1 for occlusion prediction, 0.85 NDCG@3, and 0.83 Edge F1 for connection inference, using synthetic data to avoid costly real-world annotation. The approach addresses known weaknesses of pipeline-based harvesters such as SWEEPER, whose 2018 sweet-pepper success rate was only 58%, and could potentially lift picking success from ~60% to ~80%. Stated limitations include unvalidated sim-to-real transfer, static scenes without contact-induced deformation, and single-plant scenarios.

Core Problem: The "Last Mile" Challenge for Agricultural Robots

Traditional harvesting robots follow a "detect → localize → grasp" pipeline, but face several issues:

Occlusion: fruit is often blocked by leaves
Unclear physical relations: the system doesn't know which leaves are "in the way" versus "physically connected"
Direction sensitivity: reaching from different angles gets blocked by different leaves
Dynamic environments: plants are not rigid bodies and deform on contact

SWEEPER, the best-known sweet pepper harvesting robot, achieved only a 58% success rate in 2018.

SG-DOR's Core Innovations

1. Scene Graph

Nodes: fruit, leaves, stems, branches
Edges: physical connection relations and spatial occlusion relations

2. Direction-Conditioned Occlusion Reasoning

Technical Architecture

Input: instance-segmented organ point clouds Core modules:

Scene graph construction (nodes + edges)
Occlusion ranking task
Direction-aware graph neural network (per-fruit leaf-set attention + union-level aggregation)

Output

Experimental Results

Tested on synthetic pepper datasets:

| Metric | Result | |--------|--------| | Occlusion prediction F1 | 0.73 | | NDCG@3 | 0.85 | | Connection inference Edge F1 | 0.83 |

Significance

1. From "seeing" to "understanding": solves the "what are the relations?" problem 2. Data efficiency: uses synthetic datasets, avoiding expensive real-world annotation 3. Direct application value: could potentially raise picking success rates from ~60% to ~80%

Limitations

Synthetic-to-real performance gap remains unvalidated
Static scenes only; contact-induced deformation is not modeled
Single-plant scope; multi-plant interactions are not covered

One-Sentence Takeaway

SG-DOR turns pepper picking from a geometric problem into a relation-reasoning problem—robots no longer merely "see" fruit and leaves, but "understand" their spatial and physical connections.

*Analysis date: 2026-03-09*