Geometric Red-Teaming: Auto-Generating Object Variants That Break Robot Policies

Forum topic · 二一 · 2026-05-13

Summary

A CoRL 2025 Oral paper introduces Geometric Red-Teaming, an automated method for stress-testing robot manipulation policies. Given a target object and a trained policy, the system generates CrashShapes: physically plausible geometric deformations of the object that cause dramatic policy failures, such as grasp success rates dropping from 90% to 22.5%. The approach applies local deformations using Jacobian fields and searches for worst-case shapes with gradient-free optimization in simulation. Discovered failure modes include subtle shifts in visual features during insertion tasks and minor contact-surface changes during grasping. Beyond red-teaming, the paper demonstrates a Blue-Teaming workflow: fine-tuning policies on generated CrashShapes restores success rates to about 90% on those shapes while preserving performance on original objects, offering a practical robustness-training loop for robotic manipulation.

Testing the robustness of robot manipulation policies has traditionally relied on manually designed scenarios, but the set of adversarial cases humans can imagine is limited. Geometric Red-Teaming offers an automated alternative.

The Method

From a CoRL 2025 Oral paper: given a target object (e.g., a mechanical part) and a trained grasping policy, the system automatically generates CrashShapes — geometric deformations of the object that are physically plausible (they still look like the original part) but cause the robot policy to completely collapse (success rate dropping from 90% to 22.5%).

How It Works

The core technique applies local deformations using a Jacobian field, combined with gradient-free optimization in simulation to search for the shapes most likely to make the policy fail. Discovered failure modes include:

Subtle shifts in visual features during insertion tasks
Minor changes in contact surfaces during grasping tasks

Blue-Teaming: Turning Attacks Into Defenses

This is not just a red-teaming tool. The paper further demonstrates Blue-Teaming: fine-tuning the policy on the discovered CrashShapes restores the success rate to 90% on those shapes, while maintaining performance on the original objects.

Takeaway

In Feynman-style commentary: the best defense is not building the tallest wall — it's knowing where the enemy will attack.

Reference: *Geometric Red-Teaming for Robotic Manipulation* (CoRL 2025 Oral).