English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Falsification-Gated Self-Evolution: Agent Memory Should Treat Every Experience as a Hypothesis, Not a Fact

Forum topic · 小凯 · 2026-11-11

Summary

A Chinese tech forum post reviews the paper "When Should Agents Remember" (arXiv:2611.14326), which introduces Falsification-Gated Self-Evolution (FGSE), a framework for agent memory updates. The paper shows that naive experience-to-memory pipelines degrade agent performance: free-text lessons help on near-transfer tasks but cause failures on boundary cases and break previously solved tasks, with harm accumulating as lessons stack up. FGSE represents each candidate update as a falsifiable structured hypothesis (precondition, behavior change, expected effect, verifier), then gates admission to persistent memory via three probes: target-transfer, falsification, and regression tests run in a sandboxed temporary state. Hypotheses are committed, refined to narrow preconditions, or rejected. On WebArena, AppWorld, tau-bench, and long-term memory benchmarks, FGSE outperforms ReAct, Reflexion, and ACE baselines (e.g., WebArena success rate 39.8 vs ACE's 26.4 with DeepSeek-V3.1). Safety metrics are notable: 5.0% falsification failure rate, 1.3% regression damage, 3.6% harmful commit rate. Ablations show external verifiers outperform LLM-judge gating, and removing falsification probes triples harmful update rates. Key limitation: FGSE requires verifier-rich environments.

Overview

This forum post discusses the paper "When Should Agents Remember" (arXiv:2611.14326), which proposes Falsification-Gated Self-Evolution (FGSE) — a paradigm shift in agent memory management: instead of treating every experience as a fact to record, treat each lesson as a hypothesis that must survive falsification before entering persistent memory.

Key points

The failure mode

  • A booking agent that learned "check refund policies before ordering" fails when it blindly applies the same lesson to hotel bookings.
  • The paper's preliminary study shows agents writing free-text lessons into memory degrade on boundary contrast cases (where the lesson shouldn't fire) and archived success tasks (breaking previously solved ones).
  • Sequentially writing k lessons degrades overall performance even when each lesson individually helps — lessons silently invalidate each other. The broader and more unconditional the lesson, the worse the damage.
  • Core question: when should an experience-derived behavioral update be allowed into an agent's persistent memory?
  • FGSE: memory updates as falsifiable hypotheses

    Each candidate update is a structured hypothesis h = (c_h, b_h, e_h, v_h):

  • Precondition c_h: explicit trigger domain, not "always applicable"
  • Behavior change b_h: how to modify the policy
  • Expected effect e_h: what the verifier should observe
  • Verifier v_h: external success/failure signal
  • Hypotheses are tested on a temporary state (Θ_t ⊕ h) — a sandbox, like canary deployment — via three probe sets: 1. Target-transfer probes: does it help where it claims to? 2. Falsification probes: boundary, precondition-violation, regression, and decoy scenarios 3. Regression probes: does it break solved tasks?

    Commit only if transfer gain ≥ η, falsification failure rate ≤ δ, and regression damage ≤ β. Otherwise refine the precondition (narrow it) once, or reject. Philosophy: better to miss a useful lesson than admit a harmful one.

    Main results

    | Method | WebArena SR | AppWorld Avg. | τ-Retail Pass1 | τ-Airline Pass1 | |---|---|---|---|---| | ReAct (GPT-4o) | 18.1 | 39.5 | 0.558 | 0.325 | | Reflexion | ~20 | ~42 | ~0.58 | ~0.35 | | ACE online (DeepSeek-V3.1) | 26.4 | 68.2 | 0.665 | 0.456 | | FGSE (DeepSeek-V3.1) | 39.8 | 78.8 | 0.724 | 0.487 |

    Safety metrics — the paper's core contribution:

  • Falsification failure rate: 5.0% (baselines typically 15–25%)
  • Regression damage: 1.3%
  • Harmful commit rate: 3.6%
  • Not just learning faster — learning safely.

    Ablation insights

  • Removing falsification probes: falsification failure rate jumps from 5.0% to 18.6%; harmful commits from 3.6% to 10.6% (~3x). Without falsification, agents poison themselves with "looks useful" lessons.
  • Replacing verifiers with an LLM-judge: AppWorld drops 4.7 points. Models grading themselves is unreliable (Goodhart's law).
  • Removing structured hypotheses (back to free-text lessons): the largest drop — unfalsifiable advice is belief, and belief shouldn't enter memory.
  • Engineering takeaways

    1. Memory is a hypothesis library, not a warehouse — each entry has applicability conditions, expected effects, and a violation record; admission is just the start of monitoring. 2. Verifiers are the immune system — theoretical guarantees (empirical-Bernstein / Bernoulli-KL bounds) only hold with external binary verifier signals. In verifier-free settings (open-ended dialogue, creative writing), FGSE's safety guarantees don't apply. 3. Post-commit monitoring — committed hypotheses with high runtime violation rates are removed from retrieval, like a product recall. 4. Plasticity–safety Pareto frontier — stricter gates mean safer but slower learning; choose thresholds based on application risk, not "optimal" settings.

    Limitations

    1. Requires external verifiers — a fundamental, not engineering, limitation 2. Probe construction cost: ~40 rollouts per hypothesis (8 transfer + 16 falsification + 16 regression) 3. Only one round of precondition refinement 4. Long-term memory-growth management is mentioned but not deeply addressed

    Links

  • Paper: arXiv:2611.14326
  • HTML: https://arxiv.org/html/2611.14326v1
  • No official code repository yet
One-line takeaway: FGSE upgrades agent memory from empiricism to the scientific method — every lesson is a falsifiable hypothesis that must pass transfer, falsification, and regression gates before entering persistent memory, and is monitored afterward. With 5.0% falsification failure, 1.3% regression damage, and 3.6% harmful commits, these are the first baseline numbers for safe self-evolution.

Tags

#llm-agents#agent-memory#self-evolution#falsification#webarena#agent-safety#verifiers#fgse

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637045