Paper Overview
- Field: Machine Learning (ML)
- Authors: Georgina Cosma, Axel Finke
- Published: 2026-05-28
- arXiv: 2605.27569
- M2 (oracle-comparative): Measures whether forget-set records occupy the same representational position as in a model retrained without them.
- M4 (oracle-free): Detects residual traces from the unlearned model's internal similarity structure alone, with no retraining required.
- Four approximate unlearning methods all passed output-level evaluation.
- Under a linear mixed-effects model, M2 detected significant residuals in 10 of 12 conditions (p<0.05).
- Residual effect sizes grew as the forget proportion increased.
- M4 worked as a pre-unlearning diagnostic tool across tabular data, images, clinical text, and face recognition, detecting identity-level memorization signals that no existing method fully erased.
Summary
Machine unlearning aims to remove the influence of specific training records from a deployed model without retraining from scratch. Current verification protocols check this at the output level through membership inference, retain accuracy, and forget-set accuracy. However, a model can satisfy all three criteria while still encoding the forgotten records in its intermediate representations.
RULER addresses this gap with a set of representation-level verification metrics:
Key Findings
Original Abstract (excerpt)
> Machine unlearning aims to remove the influence of specific training records from a deployed model without retraining from scratch. Current protocols verify this at the output level through membership inference, retain accuracy, and forget-set accuracy, but a model can satisfy all three whilst still encoding forgotten records in its intermediate representations. We introduce RULER, a set of representation-level verification metrics...
*Auto-collected on 2026-05-29*