Paper link: arXiv:2608.14229
Imagine trying to forget a person. A brief acquaintance is easy to forget; a close friend of ten years is nearly impossible. Language model unlearning works the same way. When you ask a model to "forget a fact," how many times that fact appeared during pretraining determines how deeply it is encoded. Rare facts appear only a few times in the corpus and are easy to remove; popular facts are encoded repeatedly and resist identical gradient pressure.
The problem: existing machine unlearning methods—Gradient Ascent (GA), Gradient Difference (GD), NPO, WGA—treat every forget sample as equally difficult. It is like scrubbing a wine glass and a cast-iron pan with the same brush: the glass breaks while the pan stays greasy.
AdaPop's core insight: forgetting difficulty scales with fact popularity, but gradient updates do not differentiate.
The Popularity Gap: A Systematic, Overlooked Failure
The paper names this phenomenon the "popularity gap":
- Rare facts are over-erased: gradient pressure is too strong, destroying capabilities the model should keep
- Popular facts are under-erased: gradient pressure is too weak; simply rephrasing a question can extract the "forgotten" answer
- ~5x leakage reduction under paraphrased queries: forgotten content resists re-extraction via rephrased prompts
- ~1.6x leakage reduction under adversarial queries: adversarial prompts designed to elicit unlearned information are far less effective
- General capability retention: MMLU and HellaSwag change by no more than 0.05 before and after unlearning
This is not a bug in one method—it afflicts all existing approaches. Krishnan et al. 2025 and Borisiuk et al. 2026 documented the phenomenon, but no prior work proposed a training-time fix.
Why can't earlier solutions fix it? The authors point out that WGA weights by model confidence, but confidence reflects "current output behavior," not "parametric encoding depth." A fact may have low confidence yet remain deeply rooted in the parameters, and vice versa. You need an external signal—popularity—to measure encoding depth.
AdaPop: External Popularity Signal + Dual Ascent Controller
AdaPop does two things:
First, it converts an external popularity signal into gradient weights. Popularity scores from Wikidata are transformed into a power-law exponent beta that weights the per-token ascent gradient signal. Popular facts get a sharper ascent signal; rare facts get a gentler one. This directly targets the root cause of the popularity gap.
Second, a dual ascent controller automatically balances forgetting and retention. The retention penalty acts as a Lagrange multiplier, dynamically adjusted each epoch based on drift in the retention loss. This removes per-dataset hyperparameter tuning and prevents retention collapse as forgetting pressure increases.
The derivation of beta is not ad hoc. Appendix B of the paper provides a full three-regime gradient analysis: beta < 0 amplifies rare facts (counterintuitive but useful), beta = 0 reduces to uniform weights (equivalent to GA), and beta > 0 amplifies popular facts. The authors calibrate empirically with three anchors plus Wikidata scores, yielding a closed-form solution.
Results: 5x Leakage Reduction
Across three models (Llama-3.1-8B, Qwen2.5-7B, Gemma-7B) on two real-world benchmarks (DUET and RWKU):
| Method | Forgetting | Retention | Problem | |--------|-----------|-----------|---------| | GA | Strong | Collapses | Language ability breaks down entirely | | GD | Weak | Asymmetric damage | Hurts both rare and popular facts | | NPO | Moderate | Stable | Cannot move deeply encoded facts | | WGA | Model-dependent | Model-dependent | Confidence ≠ encoding depth | | AdaPop | Strongest | Stable | — |
The key is not just "more thorough forgetting" but "more precise forgetting." Internal representation analysis shows that after AdaPop, hidden states of the forget set are farther from the original model than with other methods (genuine internal erasure), while hidden states of the retain set barely move (retention intact).
Another Case for the Evaluation Blind-Spot Law
AdaPop's story fits the "evaluation blind-spot" pattern perfectly. Prior unlearning evaluations used average pass rates—"how much was forgotten on average." But averages hide paired structure: over-forgetting of rare facts plus under-forgetting of popular facts can look fine on average.
This mirrors Progressive Cramming (99% token accuracy masking 100% generation failure) and the Regression Tax (average gains masking 59% regressions). The shared lesson: single metrics can mask critical failure modes.
AdaPop's contribution is not only a better unlearning method but a better evaluation lens—stratifying leakage rates by popularity reveals where forgetting actually fails. The paper's DUET paraphrase subset and RWKU adversarial-robustness subset implement exactly this popularity-stratified evaluation.
A Deeper Insight
One sentence from the paper is worth rereading: "confidence reflects current output behaviour rather than parametric encoding depth."
This points to a more general problem: the model's own signals are not the model's internal reality. Two-Process Theory says AI self-reports are artifacts of training procedures, not windows into inner states; AdaPop says model confidence is a snapshot of output behavior, not a measurement of parametric depth. Both say the same thing—do not treat a model's description of itself as the truth about its internals.
To measure encoding depth, you need an external signal (popularity); to test whether a model truly "knows" something, you need behavioral probes (residual stream directions). The model's own word does not count.
---
Paper: arXiv:2608.14229 Code: github.com/Anya-wUw/open-unlearning Full HTML text: arxiv.org/html/2608.14229v1