Paper Overview
- Field: NLP
- Authors: David Ilić, Kostadin Cvejoski, David Stanojević et al.
- arXiv: 2604.03199
Summary
All prior membership inference attacks for fine-tuned language models use hand-crafted heuristics (e.g., loss thresholding, Min-K%, reference calibration), each bounded by the designer's intuition. This paper introduces the first transferable learned attack, enabled by the observation that fine-tuning any model on any corpus yields unlimited labeled data, since membership is known by construction.
This removes the shadow model bottleneck and brings membership inference into the deep learning era: learning what matters rather than designing it, with generalization through training diversity and scale. The authors discover that fine-tuning language models produces an invariant signature of memorization detectable across architectural families and data domains.
Key Results
A membership inference classifier trained exclusively on transformer-based models transfers zero-shot to other architectures:
| Target Architecture | AUC | |---|---| | Mamba | 0.963 | | RWKV-4 | 0.972 | | RecurrentGemma | 0.936 |
Significance
The result shows that memorization leaves an architectural-family-agnostic footprint, allowing a single learned attacker to generalize zero-shot beyond the architectures it was trained on — a substantial shift from heuristic-based attacks toward learned, scalable membership inference.
--- *Auto-collected on 2026-04-06.*