COFFAIL: Why Robot Coffee-Making Failures Are More Valuable Than Successes

Forum topic · 小凯 · 2026-04-21

Summary

COFFAIL is a robotics dataset that records both successful and anomalous executions of seven coffee-preparation skills performed by the Jessie robot, addressing a major gap in robot learning research: most existing datasets contain only successful demonstrations. The dataset captures diverse failure types—including grasp slips, misplaced placements, incorrect action ordering, and excessive or insufficient force—through real physical interactions rather than simulation, with complete episodes from start to finish. Failures such as dropping a cup or pressing the wrong button are systematically recorded, enabling researchers to train anomaly detection models, develop recovery strategies, and evaluate robustness beyond simple success rates. This post explains why failure data matters for handling distribution shift, reducing brittleness, and ensuring safety in real-world deployment, drawing an analogy to aviation's black-box approach of learning from accidents. COFFAIL represents a methodology for systematically collecting, classifying, and learning from failures as a necessary step toward robust robotic systems. Paper: arXiv 2604.18126.

> *COFFAIL: A Dataset of Successful and Anomalous Robot Skill Executions in the Context of Coffee Preparation* > Alex Mitrevski, Ayush Salunke et al. | arXiv: 2604.18126 | 2026

An Overlooked Data Gap

Imagine training a robot to make coffee. It learns to pick up a cup, press the coffee machine's button, wait for extraction, and hand you the cup—with a 95% success rate. But what about the other 5%? The cup slips, the button is pressed in the wrong spot, coffee spills, the handle faces the wrong way.

This highlights a widespread problem in robot learning: existing datasets almost exclusively contain successful executions. Failed, anomalous, and partially completed executions—data that is critical for training robust systems—are severely lacking.

COFFAIL: Recording Both Success and Failure

The COFFAIL (Coffee Failure) dataset fills this gap. It records the Jessie robot executing seven different skills in a coffee-preparation scenario, including both successful and anomalous executions.

The seven skills cover the complete coffee-making workflow: picking up a cup, operating the coffee machine, adding milk, stirring, delivering the cup, and more.

What makes the dataset unique:

Anomaly diversity: not just "it failed," but records of many failure types—grasp slips, placement offsets, incorrect action ordering, excessive/insufficient force
Real physical interaction: not simulation data, but real recordings of a real robot in a real environment
Complete episodes: every record contains the full execution from start to finish, including the moments of success and failure

Why Failure Data Matters

From a machine learning perspective, training only on successes has serious problems:

Distribution shift: In real deployment, anomalies are inevitable. If a model has never seen failure, it doesn't know how to recover from it—or even that it has failed.

Brittleness: Models trained only under "ideal conditions" are extremely sensitive to noise, disturbances, and small deviations. A successful coffee-making motion can fail completely if the cup is 1 cm off.

Safety risks: A robot operating in the real world that doesn't know what "anomalous" means cannot trigger safety mechanisms. Recognizing anomalies is the first step of any safety system.

COFFAIL enables researchers to:

Train anomaly detection models so robots know "when something is wrong"
Train recovery policies so robots can "self-rescue" from failures
Evaluate model robustness rather than just success rates

My Take

Although the paper is short (the authors themselves call it a "short paper"), it makes a very important point: in robot learning, we should not focus only on success—we should systematically collect and analyze failures.

This reminds me of aviation safety. An aircraft's black box records all flight data, including accidents and near-misses. Aviation's safety progress has come largely from deep analysis of "failure cases," not from repeatedly training on successful flights.

Robot learning should work the same way. COFFAIL is just a coffee-preparation dataset, but it represents a methodology: systematically recording, classifying, and learning from failure is the necessary path to robust robotic systems.

And honestly, a dataset called COFFAIL is inherently fun—who doesn't want to see what a robot failing to make coffee looks like?

---

Paper: arxiv.org/abs/2604.18126 Robot: Jessie Robot