Overview
This forum post discusses the paper "When Should Agents Remember" (arXiv:2611.14326), which proposes Falsification-Gated Self-Evolution (FGSE) — a paradigm shift in agent memory management: instead of treating every experience as a fact to record, treat each lesson as a hypothesis that must survive falsification before entering persistent memory.
Key points
The failure mode
- A booking agent that learned "check refund policies before ordering" fails when it blindly applies the same lesson to hotel bookings.
- The paper's preliminary study shows agents writing free-text lessons into memory degrade on boundary contrast cases (where the lesson shouldn't fire) and archived success tasks (breaking previously solved ones).
- Sequentially writing k lessons degrades overall performance even when each lesson individually helps — lessons silently invalidate each other. The broader and more unconditional the lesson, the worse the damage.
- Core question: when should an experience-derived behavioral update be allowed into an agent's persistent memory?
- Precondition c_h: explicit trigger domain, not "always applicable"
- Behavior change b_h: how to modify the policy
- Expected effect e_h: what the verifier should observe
- Verifier v_h: external success/failure signal
- Falsification failure rate: 5.0% (baselines typically 15–25%)
- Regression damage: 1.3%
- Harmful commit rate: 3.6%
- Removing falsification probes: falsification failure rate jumps from 5.0% to 18.6%; harmful commits from 3.6% to 10.6% (~3x). Without falsification, agents poison themselves with "looks useful" lessons.
- Replacing verifiers with an LLM-judge: AppWorld drops 4.7 points. Models grading themselves is unreliable (Goodhart's law).
- Removing structured hypotheses (back to free-text lessons): the largest drop — unfalsifiable advice is belief, and belief shouldn't enter memory.
- Paper: arXiv:2611.14326
- HTML: https://arxiv.org/html/2611.14326v1
- No official code repository yet
FGSE: memory updates as falsifiable hypotheses
Each candidate update is a structured hypothesis h = (c_h, b_h, e_h, v_h):
Hypotheses are tested on a temporary state (Θ_t ⊕ h) — a sandbox, like canary deployment — via three probe sets: 1. Target-transfer probes: does it help where it claims to? 2. Falsification probes: boundary, precondition-violation, regression, and decoy scenarios 3. Regression probes: does it break solved tasks?
Commit only if transfer gain ≥ η, falsification failure rate ≤ δ, and regression damage ≤ β. Otherwise refine the precondition (narrow it) once, or reject. Philosophy: better to miss a useful lesson than admit a harmful one.
Main results
| Method | WebArena SR | AppWorld Avg. | τ-Retail Pass1 | τ-Airline Pass1 | |---|---|---|---|---| | ReAct (GPT-4o) | 18.1 | 39.5 | 0.558 | 0.325 | | Reflexion | ~20 | ~42 | ~0.58 | ~0.35 | | ACE online (DeepSeek-V3.1) | 26.4 | 68.2 | 0.665 | 0.456 | | FGSE (DeepSeek-V3.1) | 39.8 | 78.8 | 0.724 | 0.487 |
Safety metrics — the paper's core contribution:
Not just learning faster — learning safely.
Ablation insights
Engineering takeaways
1. Memory is a hypothesis library, not a warehouse — each entry has applicability conditions, expected effects, and a violation record; admission is just the start of monitoring. 2. Verifiers are the immune system — theoretical guarantees (empirical-Bernstein / Bernoulli-KL bounds) only hold with external binary verifier signals. In verifier-free settings (open-ended dialogue, creative writing), FGSE's safety guarantees don't apply. 3. Post-commit monitoring — committed hypotheses with high runtime violation rates are removed from retrieval, like a product recall. 4. Plasticity–safety Pareto frontier — stricter gates mean safer but slower learning; choose thresholds based on application risk, not "optimal" settings.
Limitations
1. Requires external verifiers — a fundamental, not engineering, limitation 2. Probe construction cost: ~40 rollouts per hypothesis (8 transfer + 16 falsification + 16 regression) 3. Only one round of precondition refinement 4. Long-term memory-growth management is mentioned but not deeply addressed