Summary
Machine unlearning seeks to remove the influence of specific training records from deployed models without retraining from scratch, but existing verification protocols only check output-level signals such as membership inference, retain accuracy, and forget-set accuracy. RULER introduces representation-level verification metrics that reveal hidden residuals. The oracle-comparative metric M2 measures whether forgotten records occupy the same position in representation space as in a model retrained without them, while the oracle-free metric M4 detects residuals purely from the unlearned model's internal similarity structure, requiring no retraining. Experiments show that four approximate unlearning methods all pass output-level evaluation, yet under a linear mixed-effects model, M2 detects significant residuals in 10 of 12 conditions (p<0.05), with effect sizes growing as the unlearning fraction increases. M4 works as a pre-unlearning diagnostic across tabular data, images, clinical text, and face recognition, detecting identity-level memorization signals that no existing method fully erases. Source: arXiv 2605.27569 (Georgina Cosma, Axel Finke).
Paper Overview
Field: Machine Learning
Authors: Georgina Cosma, Axel Finke
Posted: 2026-05-28
arXiv: 2605.27569
Summary
Machine unlearning aims to remove the influence of specific training records from a deployed model without retraining from scratch. Current protocols verify this at the output level through membership inference, retain accuracy, and forget-set accuracy — but a model can satisfy all three metrics while still encoding forgotten records in its intermediate representations.
RULER
RULER is a set of representation-level verification metrics:
- M2 (oracle-comparative): measures whether forget-set records occupy the same representational position as in a model retrained without them.
- M4 (oracle-free): detects residuals from the unlearned model's internal similarity structure alone, without requiring retraining.
Key Findings
- Four approximate unlearning methods all pass standard output-level evaluation.
- Under a linear mixed-effects model, M2 detects significant residuals in 10 of 12 conditions (p<0.05).
- The effect size grows as the unlearning fraction increases.
- M4 serves as a pre-unlearning diagnostic tool across tabular data, images, clinical text, and face recognition, detecting identity-level memorization signals that existing methods fail to fully erase.
Source
- Paper: arXiv:2605.27569
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177980484