Learning from Failure: How LEAFE Teaches AI to Reflect Like Humans

Forum topic · 小凯 · 2026-03-18

Summary

This article explores a key limitation of reinforcement learning: most AI systems are outcome-driven, discarding failed attempts and only reinforcing successes—much like repeatedly falling into the same pit without asking why. It introduces LEAFE (Learning Feedback-Grounded Agency from Reflective Experience), a framework that lets AI learn from failure through three steps: exploration, reflection (summarizing error feedback, backtracking to the pivotal decision point, and branching into alternative paths), and distillation of those insights into the model. In experiments on coding and agentic tasks, LEAFE consistently improved Pass@1 over base models and boosted Pass@128 by up to 14%, outperforming outcome-driven baselines like GRPO and experience-based methods like Early Experience. The author explains why reflection works: it counters distribution sharpening with distribution expansion, replaces terminal-only rewards with rich process feedback, and shifts AI from passive policy-following to active exploration. The article also draws analogies to human learning (scientific experimentation, athletic training, mental simulation), outlines applications in automated programming, research assistance, education, and robotics, and candidly discusses limitations: extra compute cost, dependence on reflection quality, and unproven cross-task generalization. Its core message: true learning is not about getting it right once, but about extracting lessons from every mistake.

An Ancient Story

Let's start with an old joke.

A man falls into the same pit every time he walks past it. When asked why he never avoids it, he replies: "I only optimize the final outcome—as long as I climb out, that's a success. How I got out doesn't matter."

Absurd, right? Yet surprisingly, this is exactly how most AI is trained today.

The Trap of Reinforcement Learning

Modern AI—especially "agents" that complete tasks autonomously—is mostly trained with reinforcement learning: try an action, receive a reward or penalty, adjust the policy, repeat.

But there's a key difference from human learning. After failing, humans think: "What did I do wrong?" "What if I had done it differently?" Most AI, by contrast, simply remembers "this action got rewarded, that action got punished," and reinforces rewarded behaviors.

> Note: This is the credit assignment problem in reinforcement learning. When an AI completes a long-horizon task and receives a reward, it must figure out which specific decision caused the success—extremely difficult in complex tasks.

This approach is called outcome-driven learning. It's simple and direct—only results matter—but it turns AI into an "outcome absolutist": as long as it succeeds, it doesn't care how, nor what failures could teach it.

A Concrete Example

Suppose an AI gets 10 attempts at a coding task. The first nine attempts all fail, each producing a valuable error message saying "this path doesn't work." In traditional outcome-driven learning, those nine failures are just noise—discarded while only the tenth success is reinforced.

But think about it: if the AI could reflect on those failures, analyzing the cause of each error, it might succeed in far fewer attempts. And the lessons from reflection can be transferred to future tasks—the AI can avoid paths already proven dead ends.

> Note: This is the essence of human learning. We don't just remember what works—we understand *why* it works and why alternatives don't. That deep understanding lets us generalize to entirely new problems.

The Birth of LEAFE

This is the problem LEAFE ("Learning Feedback-Grounded Agency from Reflective Experience") addresses. Its core idea: let AI learn from failure, not just from success.

The workflow has three steps:

Step 1: Exploration

The AI attempts the problem as usual, experiencing both successes and failures.

Step 2: Reflection

The key innovation. When the AI fails at a step, instead of abandoning that path, it:

1. Summarizes feedback: organizes error messages, warnings, and outputs into "actionable experience" 2. Backtracks: returns not to the very beginning, but to the pivotal decision point that caused the failure 3. Branches out: explores different choices from that critical point

It's like being lost in a maze: a smart person says, "Wait, I went wrong at the third junction. Let me go back there and try another route," rather than restarting from scratch.

Step 3: Distillation

LEAFE distills the insights gained from reflection—"if only I had done this instead"—into the model. Next time it faces a similar problem, it doesn't repeat the mistake; the experience is internalized.

Striking Experimental Results

The research team tested LEAFE on multiple tasks, including coding and agentic tasks (multi-step environments like web browsing and file operations).

Pass@1 Improvement

Pass@1 measures the probability of success on the first attempt. With LEAFE, Pass@1 consistently beat the base model—meaning reflection also improves the AI's "first-shot intuition," not just long-run performance.

The Pass@128 Leap

More striking is Pass@128—the probability of at least one success in 128 attempts. LEAFE improved this metric by up to 14% over traditional methods. If a traditional approach succeeds 50% of the time in 128 tries, LEAFE can reach 64%—a gap that can decide whether a project ships on time.

> Note: Pass@k measures an AI's exploratory capacity—whether it can eventually find a solution given enough attempts. High Pass@k means the AI doesn't get stuck in local optima.

Baseline Comparisons

LEAFE outperformed both traditional outcome-driven baselines (like GRPO) and other experience-based methods (like Early Experience). Its success isn't just "more data"—it's smarter use of data, especially failed experiences.

Why Does Reflection Work So Well?

From Distribution Sharpening to Distribution Expansion

Outcome-driven learning causes distribution sharpening: the AI latches onto successful behavior patterns and depends on them increasingly, like a commuter who only ever takes one shortcut—even when it's blocked someday, he can't imagine alternatives. LEAFE's reflection enables distribution expansion: the AI knows not just what works, but *why alternatives fail*, so it can explore new possibilities when familiar paths break down.

From Terminal Rewards to Process Feedback

Traditional RL cares only about the final success signal—a teacher who only grades the final exam and never marks homework. LEAFE uses all environmental feedback: error messages, warnings, intermediate results, like a teacher giving step-by-step guidance.

From Passive Acceptance to Active Exploration

Most importantly, LEAFE transforms AI from passive policy-following into active exploration—"Why did I fail just now? What other possibilities exist?" This agency is a core feature of intelligence.

A Mirror of Human Learning

LEAFE's design borrows heavily from human learning. A scientist hypothesizes, experiments, fails, *reflects* on why, adjusts, and retries. An athlete doesn't just "try again" after falling—she replays the footage, analyzes what went wrong, and adjusts. LEAFE's "backtrack + branch exploration" mirrors this.

> Note: Cognitive scientists identify "mental simulation" as a key human learning mechanism—we simulate possibilities in our minds, not just through real-world trial and error. LEAFE's reflection gives AI something like this capability.

Application Prospects

Automated programming: AI that learns from compilation errors and test failures to continuously improve code
Scientific research assistants: helping design experiments, analyze failures, suggest improvements
Personalized education: reflecting on student error patterns to give targeted feedback
Robot control: learning from failed manipulations to adapt to the real world

Limitations and Reflections

LEAFE isn't a panacea:

Compute cost: reflection requires extra computation—"thinking" about failures and simulating alternatives, problematic for real-time applications
Reflection quality: if the AI can't accurately diagnose failures or propose meaningful alternatives, reflection is useless—like a struggling student tutoring other struggling students
Generalization: whether lessons learned in task A transfer to task B needs more research. Humans can apply lessons across seemingly unrelated domains—how far is AI from that?

Conclusion: From "Getting It Right" to "Learning"

LEAFE teaches us something important: the essence of learning is not "getting it right" but "learning."

Solving a problem correctly doesn't mean mastering it. Real learning comes from reflecting, understanding, and improving after mistakes. Failure isn't the opposite of learning—it's the opportunity. Errors, failed attempts, wrong turns—they aren't noise, they're signals. The question is whether you can extract useful information from them.

Maybe one day, when we ask an AI "how did you learn this?" it will answer like a good student: "By reflecting on many failures." That's when we'll know AI has truly begun to *learn* how to learn.

References

1. LEAFE: Learning Feedback-Grounded Agency from Reflective Experience (2026). arXiv preprint. The core paper proposing the framework. 2. Schulman, J., et al. (2017). "Proximal Policy Optimization Algorithms." arXiv preprint. 3. Andrychowicz, M., et al. (2017). "Hindsight Experience Replay." *Advances in Neural Information Processing Systems*. 4. Kaelbling, L. P., & Lozano-Pérez, T. (2013). "Integrated task and motion planning in belief space." *The International Journal of Robotics Research*. 5. Lake, B. M., et al. (2017). "Building machines that learn and think like people." *Behavioral and Brain Sciences*.

*"Wisdom lies not in never erring, but in learning something from every error."*