Where Rectified Flows Leak: Mapping Membership Signals Along the Interpolation Path
> FLUX.1, Stable Diffusion 3, and VoiceBox—popular generative systems built on Rectified Flows—share a privacy vulnerability: the membership signal for training data is not uniformly distributed along the denoising interpolation path, but concentrated in a bell shape around a specific λ value. The paper not only locates this leak point but derives a closed-form formula for it.
Source: Sesmat, Meseguer-Brocal & Peeters, "Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path", arXiv:2606.07271.
Key points
- Membership signals are bell-shaped over λ. Unlike diffusion models, where signals spread over discrete timesteps, Rectified Flows learn a velocity field over continuous linear interpolation paths \(X_\lambda = (1-\lambda)X_0 + \lambda X_1\). The training-test loss gap concentrates near a predictable peak.
- Closed-form peak location. Loss decomposition isolates a cross-correlation term \(G_n^{\text{train}}(\lambda)\) that is zero in expectation for test data but nonzero for training data—this is the membership signal. Under isotropic Gaussian assumptions, the signal peaks at \(\lambda^* = \sigma_0^2/(\sigma_0^2+\sigma_1^2)\) (i.e., 0.5 when noise and data variances match), where the irreducible variance is twice its boundary value and the linear optimal predictor vanishes.
- Standard metrics miss it. Global loss averages over λ and dilutes the signal; validation loss keeps decreasing normally while the train-test gap at λ* grows silently from the very first epoch.
- Setup: DiT-style Transformers (140M–880M params), latents from Music2Latent / Stable Audio VAE / SD VAE; datasets MAESTRO v3, MTG-Jamendo, FMA Large, CelebA.
- Gaussianity holds for audio (low skew/kurtosis/correlation); CelebA + SD VAE violates assumptions, yet the bell shape persists—only the peak position deviates from prediction (predicted 0.45, observed 0.6–0.7).
- Ablations: peak position follows data/noise geometry (\(\Sigma_0, \Sigma_1\)); capacity and architecture affect magnitude only:
- Model size 140M → 410M → 880M: signal 0.06 → 0.09 → 0.12
- Transformer 0.09 vs UNet 0.01 (9x stronger)
- Log-normal λ sampling 0.09 vs uniform 0.06
- Smaller datasets yield stronger signals (MAESTRO strongest)
- Linear/nonlinear competition: the Transformer-vs-OLS test-loss ratio is maximal exactly at the signal peak, confirming that nonlinear capacity is exploited precisely where linear predictors fail.
- Targeted defense: concentrate privacy mechanisms (differential privacy, regularization) near λ* instead of uniformly along the path.
- Reflow as mitigation: one reflow step keeps the bell shape but sharply attenuates its magnitude.
- Efficiency–privacy trade-off: concentrating \(p(\lambda)\) near 0.5 improves convergence (as in SD3) but also amplifies membership leakage—the exact point where prediction is hardest is where the model memorizes most.
- Closed-form peak prediction requires approximate Gaussian isotropic latents.
- Independent-coupling assumption excludes reflow (preliminary experiments only).
- White-box threat model (forward-pass access); unconditional generation only; scale limited to 880M parameters.
Experimental validation
Membership inference attack
Given a query sample, the attacker samples K=100 noise vectors, reconstructs \(\hat{x}_1\) via a single forward prediction at 11 λ points, and feeds the 11-dimensional profile to an MLP:
| Method | AUC | |---|---| | Naive (single point λ*) | 0.67 | | SecMI (adapted) | 0.72 | | PIA (adapted) | 0.83 | | λ-profile MLP | 0.91 |
Only forward passes are required—no gradients or weight access.
Defenses and trade-offs
Limitations
Conclusion
The paper's core contribution is the first "memory leak map" for Rectified Flows: membership signals follow a universal bell-shaped curve over λ with a peak dictated by data geometry, accumulate silently, grow with model size, and are invisible to standard diagnostics. A leak map, the author argues, is worth more than a hundred attack recipes.