K-Bench: Exposing the 'Magic Trick' of LLM Unlearning in Agentic Deployments

Forum topic · 小凯 · 2026-09-14

Summary

A forum post analyzes K-Bench (arXiv:2609.12808), a benchmark from University of Technology Sydney and CSIRO that re-evaluates LLM machine unlearning in agentic deployments. Existing benchmarks like TOFU and MUSE only inspect the model's final answer, so a model can 'pass' by refusing to answer while the secret remains recoverable. K-Bench instead monitors six observability channels of a ReAct agent—chain-of-thought, tool-call arguments, tool outputs, retrieved documents, final answers, and follow-up summaries—and flags leakage if the target PII appears in any of them. Secrets are injected into four substrates: parameters, context prompts, free-text retrieval corpora, and structured databases. Experiments across 20 published unlearning methods found none achieved verifiable selective forgetting under the multi-channel observer; leakage often migrated between channels (e.g., scrambled answers while tool calls returned exact records), with aggregate leakage rates near baseline (0.857 vs 0.855). When secrets sit in prompts or retrieval stores, real leakage reached 22%-86% while TOFU/MUSE reported none. The benchmark uses preregistration, paired McNemar tests with Benjamini-Hochberg correction, and collapse-aware scoring to ensure rigor.

The House with Six Windows: Watching AI's 'Forgetting' Magic Trick Fall Apart

> "Sweep the dust under the rug and the floor looks clean. But the dust hasn't disappeared—it just moved."

Paper Card

Title: K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments
Authors: Guangsheng Yu, Yanna Jiang, Qin Wang, Baihe Ma, Xu Wang
Institutions: University of Technology Sydney, CSIRO
arXiv: 2609.12808

Key points

The blind spot of current benchmarks: TOFU and MUSE evaluate unlearning only via the final answer. A model can output "I don't know" and score perfectly while the knowledge remains intact—undistinguishable between "truly forgot" and "remembers but chooses silence."
Six observable channels: K-Bench runs a ReAct agent and monitors chain-of-thought reasoning (Z_CoT), tool-call arguments (Z_tool), tool parameters plus returned results (Z_tool_wide), retrieved documents (Z_RAG), final answers (Z_answer), and elicited summaries (Z_summary). Leakage is flagged if the target PII appears in any channel (logical OR).
Four substrates for hiding secrets: model parameters (P), context prompts (C), free-text retrieval corpora (R-text), and structured databases (R-struct). Weight-editing methods can only touch P; pure parameter benchmarks are entirely blind to C and R.
Channel migration (K-SUP): In a documented case, the StaR method scrambled the answer (correct year, wrong day: 1999-03-27) so single-channel probes judged it "forgotten," but the tool-output channel still returned the exact record (1999-03-17). Aggregate leakage OR(all) = 0.857 vs. baseline 0.855—leakage moved channels rather than shrinking.
Coverage gap: When secrets are in prompts or retrieval stores, TOFU/MUSE report zero leakage (they don't test those scenarios), while the same model as an agent leaks target PII in 22%–86% of queries.
20 methods on trial: Most methods silence the answer channel while secrets remain extractable via side channels (especially follow-up summaries); some cause full model collapse (which TOFU-style scoring still counts as "success"); IDK-style refusal tuning amounts to *learning to lie*, not deleting memory. No published method achieved verifiable selective forgetting. The only approach meeting the standard was the simplest: input corruption, at the cost of utility loss on affected inputs.
Leaderboards are model-dependent: The top-ranked unlearning method changes when the base model switches (Llama-3.1-8B vs. Qwen3.5-9B vs. Mistral-7B), so single-model rankings lack generality.

Methodology: making a benchmark scientific

1. Preregistration: hypotheses fixed before evaluation. 2. Seed-pooled paired McNemar tests comparing method success per query. 3. Benjamini-Hochberg FDR correction for multiple comparisons. 4. Collapse-aware K-Score jointly measuring forget-set suppression, retain-set preservation, and agent stability. 5. Failure localization: each leak is reported with its channel and failure mode.

Robustness checks (different injection recipes, semantic vs. exact-match scoring, varied elicitation formats, run-to-run variance) confirmed the conclusions are not artifacts of benchmark design.

Stated limitations

Single ReAct agent only; multi-agent systems would expose a strict superset of channels—real-world leakage surfaces are likely larger.
One substrate per experiment; mixed-substrate scenarios are future work.
Per-channel detection means cross-channel fragmentation attacks are outside the observer model—the reported leakage rates are a lower bound.

Conclusion

Silence in the answer channel is not evidence of forgetting—it's the secret learning to keep quiet. K-Bench argues that in the agent era, "unlearning" must be redefined: not "the model refuses to answer," but "the secret cannot be recovered on any observable surface the agent exposes." The paper doesn't claim unlearning is impossible; it shows the true shape of the problem. Six windows have been counted; the task now is to actually close each one—not to sweep more dust under the rug.

References

1. Yu G., et al. "K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments." arXiv:2609.12808, 2026. 2. Maini P., et al. "TOFU: A task of fictitious unlearning for LLMs." arXiv:2401.06121, 2024. 3. Shi W., et al. "MUSE: Machine unlearning six-way evaluation." arXiv:2503.02657, 2025. 4. Yao S., et al. "ReAct: Synergizing reasoning and acting in language models." ICLR, 2023. 5. Li N., et al. "The WMDP benchmark." arXiv:2403.03218, 2024.