LACUNA: First Parameter-Level Testbed Reveals SOTA LLM Unlearning Methods Are Mostly Pretending to Forget

Forum topic · ✨步子哥 · 2026-07-03

Summary

LACUNA is a testbed from Mila and McGill University for evaluating localization precision in LLM unlearning at the parameter level. By injecting synthetic PII into predefined parameter subsets of OLMo2 1B and OLMo3 7B models via masked continual pretraining, LACUNA creates ground-truth masks of where knowledge is stored. This enables measuring whether unlearning methods actually modify the weights that encode target knowledge. The results are striking: SimNPO achieves a localization ROC AUC of only 0.515—near random—while MemFlex and AlphaEdit also show poor localization, and models unlearned with them leak PII again after light fine-tuning (resurfacing attacks). In contrast, OracleGrad, a simple gradient-difference baseline given access to the ground-truth mask, reaches 0.915 AUC and resists resurfacing. The authors conclude that current SOTA methods perform obfuscation rather than erasure: they suppress outputs without removing knowledge from parameters. LACUNA shifts unlearning evaluation from output-level benchmarks to parameter-level verification, with implications for GDPR compliance, alignment, backdoor removal, and jailbreak defense. Paper: arxiv.org/abs/2607.02513; code: github.com/McGill-NLP/LACUNA.

LACUNA: You Think the Model Forgot — It's Just Pretending. The First Parameter-Level Unlearning Testbed Exposes SOTA Hallucinations

> Paper: LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning > Authors: Matteo Boglioni, Thibault Rousset, Siva Reddy, Marius Mosbach, Verna Dankers > Institutions: Mila – Quebec AI Institute, McGill University > Link: https://arxiv.org/abs/2607.02513 > Code: https://github.com/McGill-NLP/LACUNA

---

A Chilling Scenario

Imagine you're the security lead at an AI company. A user invokes their GDPR "right to be forgotten" and asks your LLM to delete a phone number that ended up in the training data. Your engineers run a SOTA unlearning algorithm, test it, and confirm the model no longer voluntarily outputs that number. You breathe a sigh of relief and sign off on the compliance report.

Three months later, someone fine-tunes your model — just a few extra steps on public PII data — and the "forgotten" phone number resurfaces.

This isn't hypothetical. It's the actual result of the resurfacing attack described in the LACUNA paper.

Where does the problem lie? In the word "forget" itself. You think the model forgot; it may just be pretending — suppressing output-layer probabilities while the weights storing that knowledge remain untouched.

The Core Problem: Output Layer ≠ Parameter Level

Existing unlearning evaluation has a huge blind spot: it only looks at outputs. If the model stops saying it, unlearning is deemed successful. But "not saying" has two possible causes:

1. Truly forgotten: the weights storing the knowledge were modified; the knowledge is erased from parameters 2. Faking it: the weights barely moved; only the output layer was tuned not to generate that text. A light fine-tuning brings the knowledge back

It's like a student answering an exam question correctly — you don't know whether they truly understand it or just memorized the answer. You need to look inside their head.

That's exactly what LACUNA does: open up the model's brain and give unlearning precision a parameter-level checkup.

How LACUNA Works: GPS for Knowledge

LACUNA's core innovation is ground-truth parameter-level localization.

The traditional dilemma: you don't know which weights store a piece of knowledge, so you can't judge whether an unlearning algorithm modified the right places. LACUNA's clever solution — since you can't know where real knowledge lives, inject your own batch of knowledge and precisely control where it's stored.

The pipeline has three phases:

Phase 1: Injection. LACUNA constructs PII for a batch of synthetic personas (names, emails, phone numbers, etc.) and injects it into predefined parameter subsets of OLMo2 1B and OLMo3 7B models using a technique called masked continual pretraining. The key is the mask — it records "which weights participated in storing this PII," like a GPS tracker attached to each piece of knowledge.

Phase 2: Unlearning. Existing SOTA unlearning methods (MemFlex, AlphaEdit, SimNPO) attempt to forget this PII while retaining other knowledge.

Phase 3: The Checkup. This is LACUNA's killer feature. With the ground-truth mask, you can directly compute localization precision — how much do the weights modified by the unlearning algorithm overlap with the weights that actually store the PII? Measured with ROC AUC: 1.0 is perfect localization, 0.5 is random guessing, <0.5 means it's deliberately modifying irrelevant parameters.

Bursting the Bubble: SOTA Localization Is Near Random

The results are shocking.

| Method | Type | Localization Precision (AUC) | |--------|------|------------------------------| | SimNPO | gradient-based | 0.515 | | MemFlex | localize + unlearn | low | | AlphaEdit | localize + unlearn | low | | OracleGrad | ground-truth assisted | 0.915 |

SimNPO's AUC of 0.515 is indistinguishable from randomly modifying parameters. In other words, this SOTA method — best-in-class on output-level benchmarks — is effectively changing weights aimlessly across the whole model, coincidentally suppressing output probabilities, while barely touching the parameters that store the knowledge.

MemFlex and AlphaEdit are even worse. They claim to "localize first, then unlearn," but LACUNA's ground-truth masks reveal their localization is largely unreliable.

Analogy: it's like hiring a demolition crew to remove one wall in a building, and they chip away a little at every wall. From outside, that wall does look shorter (the output changed) — but the building's structural integrity is ruined, and the load-bearing part of that wall (the parameters storing the knowledge) was never removed.

The Resurfacing Attack: The Cost of Pretending

If unlearning is just faking, fine-tuning can bring the knowledge back.

LACUNA ran resurfacing attacks: instruction fine-tuning on unlearned models, then checking whether PII from the forget set leaks again. Results:

MemFlex and AlphaEdit: highly vulnerable; most PII in the forget set can be re-extracted
SimNPO: slightly better, but some PII still resurfaces
OracleGrad: most resistant, with the lowest leakage rate

This directly demonstrates: higher localization precision → more thorough unlearning → more resistance to resurfacing attacks. OracleGrad's AUC is 0.915; it truly modified the right places, so the knowledge was erased, and fine-tuning can't recover it.

OracleGrad: A "Cheating" Control Group

OracleGrad is not a new method proposed by the paper. It's a thought-experiment control: feed the ground-truth mask directly to the simplest gradient-based unlearning algorithm (Gradient Difference: gradient ascent on the forget set, gradient descent on the retain set), allowing it to modify only weights inside the mask.

This "cheating" simple method crushes all SOTA methods across three dimensions: unlearning effectiveness, utility preservation, and resistance to resurfacing attacks.

The paper's subtext is clear: the problem isn't the unlearning algorithm — it's localization. If you know where to modify, even the plainest gradient method suffices. SOTA methods don't fail because their unlearning algorithms are bad; they fail because they don't know where to modify.

Why This Matters

1. A Paradigm Shift in Unlearning Evaluation

LACUNA upgrades unlearning evaluation from "looking at outputs" to "looking at parameters." It's like medicine moving from "reading symptoms" to "reading imaging" — a patient without a fever doesn't mean the infection is gone; you need a CT scan to see whether the lesion remains.

2. Real GDPR Compliance Risk

If you're a DPO at a company, you should worry. Your unlearning vendor may pass output-level tests while user PII still lives in the parameters. A fine-tune, a round of continued training, even prompt engineering could resurface that data. LACUNA provides the first tool that can genuinely verify "parameter-level forgetting."

3. Broader Implications for AI Safety

Unlearning is only one subproblem of AI safety. The pattern LACUNA reveals — output-level evaluation can mask parameter-level reality — may apply more broadly:

Alignment evaluation: a model not saying harmful things ≠ harmful tendencies eliminated
Backdoor removal: normal behavior on normal inputs ≠ backdoor removed
Jailbreak defense: refusing known jailbreaks ≠ understanding jailbreaks

Any safety evaluation that only looks at outputs can be fooled by pretending.

An Honest Assessment

LACUNA has limitations:

1. Synthetic vs. real data: LACUNA injects synthetic PII; in real-world training, PII is naturally scattered and may be stored in more distributed, entangled locations 2. Model scale: only 1B and 7B models were tested; knowledge storage patterns at tens-of-billions scale may differ 3. Mask design: the fraction of parameters covered by the mask affects conclusions; the paper discusses mask coverage effects

But these are secondary. LACUNA's core contribution — the first parameter-level ground truth for unlearning evaluation — is enough to change the field's evaluation paradigm.

Conclusion

The most profound idea in the LACUNA paper isn't a number, but a distinction: erasure vs. obfuscation.

Real forgetting is erasure — the knowledge vanishes from parameters and no attack can recover it. Fake forgetting is obfuscation — the knowledge is still there; only the output layer has been tuned to stay quiet.

LACUNA tells us: all current SOTA unlearning methods remain at the obfuscation stage. They teach the model to shut up, not to actually forget.

The next time someone tells you "our model has successfully forgotten that data," ask one question: how do you know it forgot — rather than just pretending?

---

Paper: https://arxiv.org/abs/2607.02513 Code: https://github.com/McGill-NLP/LACUNA