AI's Lesson in Reflection: When Machines Learn to Grow from Setbacks
*(Full translation of a zhichai.net explainer post)*
An Old Story
Let's start with an old joke.
A man falls into a pit every time he passes it. A passerby asks: "Why don't you avoid it?" He replies: "Because I only optimize the final outcome — if I can climb out, that's success. How I climbed out doesn't matter."
Sounds absurd, right?
Surprisingly, this is exactly how most AI is trained today.
The Trap of Reinforcement Learning
Modern AI agents are mostly trained with reinforcement learning — trial-and-error learning:
- The AI tries an action
- The environment gives feedback (reward or penalty)
- The AI adjusts its policy
- Repeat until an optimal policy is found
- Attempt 1: code written, run, error message received
- Attempt 2: code modified, run, another error
- ...
- Attempt 10: finally correct, reward given
- Summarizes feedback: organizes error messages, warnings, and outputs into "actionable experience"
- Backtracks: returns to the earlier decision point that caused the failure — not the very beginning, but the critical fork
- Branches: explores different choices from that point
- Scientists: hypothesize, experiment, get unexpected failure, reflect on why, adjust, retry. LEAFE mimics exactly this loop.
- Athletes: falling on the first try isn't followed by blindly "trying again" — they replay footage, analyze the mistake, adjust. LEAFE's "backtrack + branch exploration" is like mental replay and retry.
- Automated coding: AI that learns from compile errors and test failures to keep improving code quality
- Research assistants: AI that designs experiments, analyzes failures, suggests improvements — a true "research partner"
- Personalized education: AI that reflects on its teaching strategy based on students' error patterns
- Robot control: robots that learn from failed manipulations to adapt to the real world
- Compute cost: Reflection requires extra computation — "thinking" about failures and "simulating" alternatives, which may be problematic in real-time applications.
- Reflection quality: LEAFE depends on reflection being accurate. If the AI can't correctly summarize failure causes or propose meaningful alternatives, reflection is useless — like a struggling student tutoring other struggling students.
- Generalization: Whether reflection lessons learned in task A transfer to task B needs more research. Humans can apply lessons across seemingly unrelated domains; how far is AI from that?
Humans also learn this way, but with a key difference: after failure, humans think — "What did I do wrong?" "What if I had tried something else?" "How is this problem similar to the last one?"
Most AI simply memorizes "this got a reward, that got a penalty," and reinforces rewarded behaviors.
> Note: This is the credit assignment problem in reinforcement learning. When an AI completes a long task and gets a reward, it must figure out which specific decision caused the success — extremely difficult in complex tasks.
This approach is called outcome-driven learning. It's simple and direct, but it turns AI into an "outcome absolutist" — success by any means, and nothing learned from failure.
A Concrete Example
Suppose an AI must write a program, with 10 attempts allowed.
In traditional outcome-driven learning, only the tenth attempt matters. The nine failures are "noise" — discarded, while the success is reinforced.
But those nine failures contained nine pieces of valuable feedback, each saying "this path is a dead end." If the AI could reflect on them, it might succeed sooner. More importantly, these reflected lessons can be transferred to future tasks — next time, the AI can skip proven dead ends.
> Note: This is the essence of human learning: not just remembering "what works," but understanding "why it works" and "why alternatives don't." That deep understanding lets us generalize to brand-new problems.
The Birth of LEAFE
This is the problem LEAFE solves. Its full name: "Learning Feedback-Grounded Agency from Reflective Experience."
Core idea: let AI learn from failure, not just from success.
The workflow:
1. Exploration: The AI attempts the task as usual, experiencing successes and failures. 2. Reflection (the key innovation): When the AI fails at a step, it doesn't abandon the path immediately. It:
Like someone navigating a maze: a smart person says, "Wait, I took the wrong turn at the third junction. Let me go back there and try another path," instead of restarting from scratch. 3. Distillation: The insights from reflection — "if only I had done this" — are distilled into the model. Next time, the AI has internalized the experience and can choose better directly.
Striking Experimental Results
The team tested LEAFE on coding tasks and multi-step agentic tasks (web browsing, file operations).
Pass@1 improvements: With LEAFE, the AI's Pass@1 (first-attempt success rate) consistently beats the base model. Reflection improves not just long-horizon performance but also "first-guess intuition."
Pass@128 leap: More striking is Pass@128 — the probability of at least one success in 128 attempts. LEAFE improves this by up to 14% over traditional methods. If a traditional method succeeds 50% of the time in 128 tries, LEAFE reaches 64% — a gap that can decide whether a project ships on time.
> Note: Pass@k measures an AI's "exploration ability" — given enough attempts, can it eventually find a solution? High Pass@k means the AI doesn't get stuck in local optima.
Baselines: LEAFE beats not only outcome-driven baselines (like GRPO) but also other experience-based methods (like Early Experience). Its success comes not from "more data" but from using data smarter — especially failed experiences.
Why Does Reflection Work So Well?
From distribution sharpening to distribution expansion: Traditional outcome-driven learning suffers from distribution sharpening — the AI over-relies on learned successful patterns, like someone who found a shortcut to work and can't think of alternatives when the road is blocked. LEAFE's reflection achieves distribution expansion: the AI knows not only what works but why alternatives fail, so it can explore new possibilities when familiar paths break down.
From terminal rewards to process feedback: Traditional RL only cares about the final success signal — like a teacher grading only the final exam, never homework. LEAFE uses all feedback: error messages, warnings, intermediate results along the way. It's like having a teacher who comments at every step.
From passive acceptance to active exploration: Most importantly, LEAFE shifts AI from passive to agentic — actively asking "why did I fail?" and "what else is possible?" This agency is a core feature of intelligence.
Mirroring Human Learning
LEAFE's design borrows heavily from how humans learn:
> Note: Cognitive scientists call this "mental simulation" — we try things not just in reality but in our heads. LEAFE's reflection gives AI a form of this ability.
Application Prospects
Limitations and Caveats
LEAFE isn't a silver bullet:
Conclusion: From "Getting It Right" to "Learning"
LEAFE teaches us something important: the essence of learning is not "getting it right" but "learning."
Solving a problem doesn't mean you've learned the concept. Real learning is reflecting, understanding, and improving after getting it wrong. This holds for AI and for humans alike.
We often chase "success on the first try" and fear failure. But LEAFE reminds us: failure isn't the opposite of learning — it's the opportunity. Error messages, failed attempts, wrong turns — they aren't noise, they're signals. The question is whether you can extract useful information from them.
Maybe someday, when we ask an AI "how did you learn this?" it will answer like a good student: "I reflected on many failures and figured it out." At that point, we'll know AI has truly begun to learn how to learn.
References
1. LEAFE: Learning Feedback-Grounded Agency from Reflective Experience (2026). arXiv preprint. 2. Schulman, J., et al. (2017). "Proximal Policy Optimization Algorithms." arXiv preprint. 3. Andrychowicz, M., et al. (2017). "Hindsight Experience Replay." *Advances in Neural Information Processing Systems*. 4. Kaelbling, L. P., & Lozano-Pérez, T. (2013). "Integrated task and motion planning in belief space." *The International Journal of Robotics Research*. 5. Lake, B. M., et al. (2017). "Building machines that learn and think like people." *Behavioral and Brain Sciences*.
*"Wisdom lies not in never making mistakes, but in learning something from every mistake."*