> Paper: LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning > Authors: Matteo Boglioni, Thibault Rousset, Siva Reddy, Marius Mosbach, Verna Dankers > Institutions: Mila – Quebec AI Institute, McGill University > Link: https://arxiv.org/abs/2607.02513 > Code: https://github.com/McGill-NLP/LACUNA
A Chilling Scenario
Imagine you are the security lead at an AI company. A user invokes their GDPR "right to be forgotten" and asks your LLM to delete a phone number embedded in its training data. Your engineers run a state-of-the-art (SOTA) unlearning algorithm, test it, and confirm the model no longer outputs that number. You sign off on the compliance report.
Three months later, someone fine-tunes your model—just a few steps on public PII data—and the "forgotten" phone number resurfaces.
This is not hypothetical. This is the resurfacing attack demonstrated in the LACUNA paper.
The problem lies in the word "unlearning" itself. The model may not have actually forgotten—it may just be pretending: output probabilities are suppressed, but the weights storing that knowledge remain untouched.
Core Problem: Output Layer ≠ Parameter Layer
Current unlearning evaluation has a huge blind spot: it only looks at outputs. If the model stops saying it, the unlearning is deemed successful. But "not saying" has two possible causes:
1. True forgetting: the weights storing the knowledge were modified; the knowledge is erased from parameters. 2. Pretending: weights barely changed; only the output layer was tuned to stop generating that text. A light fine-tune makes the knowledge "resurface."
It's like a student answering an exam question correctly—you can't tell whether they truly understand or memorized the answer. You need to look inside their head.
That's exactly what LACUNA does: opens up the model and performs a parameter-level check-up of unlearning precision.
How LACUNA Works: GPS for Knowledge
LACUNA's core innovation is ground-truth parameter-level localization.
The traditional dilemma: you don't know which weights store a given piece of knowledge, so you can't judge whether an unlearning method modified the right places. LACUNA's clever solution: since you can't find where real knowledge lives, inject your own knowledge and precisely control where it's stored.
The pipeline has three phases:
Phase 1: Injection. LACUNA constructs synthetic personas with PII (names, emails, phone numbers, etc.) and injects this PII into predefined parameter subsets of OLMo2 1B and OLMo3 7B models using a technique called masked continual pretraining. The key is the mask—it records "which weights participated in storing this PII," acting like a GPS tracker for each fact.
Phase 2: Unlearning. Existing SOTA unlearning methods (MemFlex, AlphaEdit, SimNPO) attempt to forget this PII while retaining other knowledge.
Phase 3: The check-up. This is LACUNA's killer feature. With the ground-truth mask, you can directly compute localization precision—how much do the weights changed by the unlearning method overlap with the weights that actually store the PII? Measured via ROC AUC: 1.0 is perfect localization, 0.5 is random modification, <0.5 means deliberately editing irrelevant parameters.
Bursting the Bubble: SOTA Methods Localize Near Randomly
The results are startling.
| Method | Type | Localization Precision (AUC) | |--------|------|------------------------------| | SimNPO | Gradient-based | 0.515 | | MemFlex | Localize + unlearn | Low | | AlphaEdit | Localize + unlearn | Low | | OracleGrad | Given ground truth | 0.915 |
SimNPO's AUC of 0.515 is indistinguishable from randomly modifying parameters. In other words, this SOTA method—which performs best on output-level metrics—is effectively changing weights aimlessly across the whole model, coincidentally suppressing output probabilities while leaving knowledge-storing parameters almost untouched.
MemFlex and AlphaEdit are worse: they claim to "localize first, then unlearn," but LACUNA's ground-truth masks reveal their localization is largely unreliable.
Analogy: It's like hiring a demolition crew to remove one wall in a building, and instead they chip at every wall a little. From outside, that wall looks shorter (the output changed), but the building's structural integrity is damaged—and the load-bearing part of that wall (the knowledge-storing parameters) was never removed.
Resurfacing Attacks: The Cost of Pretending
If unlearning is just pretending, a brief fine-tune can make the knowledge resurface.
LACUNA ran resurfacing attacks: after unlearning, the model underwent instruction fine-tuning, then was tested for re-leaking PII from the forget set. Results:
- MemFlex and AlphaEdit: highly vulnerable; most PII from the forget set could be re-extracted
- SimNPO: somewhat better, but some PII still recoverable
- OracleGrad: most robust, with the lowest leakage rate
- Alignment evaluation: a model not emitting harmful content ≠ harmful tendencies removed
- Backdoor removal: normal behavior on normal inputs ≠ backdoor eliminated
- Jailbreak defense: refusing known jailbreaks ≠ understanding why they work
This directly demonstrates: the higher the localization precision, the more thorough the unlearning, the harder to breach via resurfacing attacks. OracleGrad's 0.915 AUC means it genuinely modified the right places, so the knowledge was erased and fine-tuning cannot recover it.
OracleGrad: A "Cheating" Control Group
OracleGrad is not a new method proposed by the paper—it's a thought-experiment control: feed the ground-truth mask to a simple gradient unlearning algorithm (Gradient Difference: gradient ascent on the forget set, gradient descent on the retain set), allowing it to modify only weights inside the mask.
This "cheating" but simple method crushes all SOTA methods across three dimensions: unlearning effectiveness, utility retention, and resistance to resurfacing attacks.
The paper's subtext is clear: the problem is not the unlearning algorithm itself—it's localization. If you know what to modify, a naive gradient method suffices. SOTA methods' weakness is not their unlearning algorithms; it's that they have no idea where to modify.
Why This Matters
1. A Paradigm Shift in Unlearning Evaluation
LACUNA upgrades unlearning evaluation from "looking at outputs" to "looking at parameters." It's like medicine moving from "observing symptoms" to "imaging"—a patient without a fever doesn't mean the infection is gone; you check the CT scan.
2. Real GDPR Compliance Risks
If you're a company's DPO (Data Protection Officer), you should be worried. Your unlearning service may pass output-level tests while user PII remains in the parameters. One fine-tune, one continued training run, or even prompt engineering could re-expose that data. LACUNA provides the first tool that genuinely verifies "parameter-level forgetting."
3. Broader Implications for AI Safety
Unlearning is just one subfield of AI safety. The pattern LACUNA reveals—output-level evaluation masks parameter-level reality—may apply more broadly:
An Honest Assessment
LACUNA has limitations:
1. Synthetic vs. real data: LACUNA injects synthetic PII, while in real-world training PII is naturally scattered across data—storage locations may be more distributed and entangled. 2. Model scale: only 1B and 7B models were tested; knowledge storage patterns in tens-of-billions-parameter models may differ. 3. Mask design: the fraction of parameters covered by the mask affects conclusions; the paper discusses mask coverage effects.
But these are secondary. LACUNA's core contribution—the first parameter-level ground truth for unlearning evaluation—is enough to change the field's evaluation paradigm.
Conclusion
The deepest takeaway from LACUNA is not a number but a distinction: erasure vs. obfuscation.
True forgetting is erasure—knowledge disappears from the parameters, unrecoverable by any attack. Fake forgetting is obfuscation—the knowledge remains; only the output layer has been tuned into silence.
LACUNA tells us: all current SOTA unlearning methods are still in the obfuscation stage. They teach models to shut up, not to truly forget.
The next time someone tells you "our model has successfully forgotten this data," ask one question: how do you know it forgot, rather than pretended?
---
Paper: https://arxiv.org/abs/2607.02513 Code: https://github.com/McGill-NLP/LACUNA