Where Rectified Flows Leak Training Data: A Membership-Signal Map Along the Interpolation Path

Forum topic · 小凯 · 2026-06-12

Summary

A forum post discusses recent research (arXiv:2606.07271, Sesmat et al.) showing that Rectified Flow generative models — the framework behind FLUX.1, Stable Diffusion 3, VoiceBox, and Stable Audio Open — leak training-data membership signals in a structured way. The paper decomposes the training loss and shows that a cross-correlation term, nonzero only for training samples, constitutes the membership signal. Theoretically, under Gaussian isotropic assumptions, the signal follows a bell-shaped curve over the interpolation parameter λ, peaking at λ* = σ0² / (σ0² + σ1²), the point of minimal linear information. Experiments on MAESTRO v3, MTG-Jamendo, FMA Large, and CelebA with DiT-style Transformers (140M–880M parameters) confirm the predictions. An 11-point λ-profile MLP classifier reaches 0.91 AUC for membership inference using only forward passes. Larger models amplify the signal (0.06→0.12), Transformers leak about 9× more than UNets, and larger datasets attenuate it. Standard validation metrics miss the leakage because the train–test gap accumulates silently. The post highlights defenses: targeted regularization near λ* and reflow, plus an efficiency–privacy tradeoff since sampling p(λ) near λ* speeds training but magnifies leakage.

Background

Generation models can leak training data beyond verbatim copying, through *membership signals* — measurable asymmetries in how the model treats trained vs. unseen samples. While prior work (Matsumoto et al. 2023; SecMI, Duan et al. 2023; PIA, Kong et al. 2023) located such signals in diffusion models' intermediate timesteps, Rectified Flows — used by FLUX.1, Stable Diffusion 3, VoiceBox, and Stable Audio Open — learn a velocity field over a linear interpolation path $X_\lambda = (1-\lambda)X_0 + \lambda X_1$ . Where the signal concentrates on this continuous path was an open question, addressed by Sesmat, Meseguer-Brocal, and Peeters in arXiv:2606.07271.

Key points

Loss decomposition: For training samples, $\mathcal{L}_{\text{train}}(\lambda) = E_n^{\text{train}}(\lambda) + \hat\sigma_n^2(\lambda) - 2G_n^{\text{train}}(\lambda)$ , where the cross-correlation term $G_n^{\text{train}}(\lambda)$ is provably zero in expectation on test data but a priori nonzero on training data — i.e., it *is* the membership signal.
Closed-form peak: Under isotropic Gaussian assumptions, the expected signal is $\mathbb{E}[G_n^{\text{train}}(\lambda)] = \sigma^2_{\text{irr}}(\lambda)\cdot\frac{n-1}{n(n-2)}$ , peaking at

\lambda^* = \frac{\sigma_0^2}{\sigma_0^2 + \sigma_1^2}

(equal to 1/2 when $\sigma_0 = \sigma_1$ ). At $\lambda^*$ , the irreducible variance is $d(\sigma_0^2+\sigma_1^2)$ — twice the boundary values — and the linear optimal predictor vanishes, forcing the model to rely on nonlinear features that cannot distinguish generalizing residuals from training-specific noise.

Silent accumulation: Global loss averages over λ and dilutes the bell-shaped signal. Validation loss keeps decreasing until early stopping, while the train–test gap at $\lambda^*$ grows from the first epoch — the model leaks before any visible overfitting.

Experimental validation

Setup: MAESTRO v3, MTG-Jamendo, FMA Large (audio), CelebA (images); latent encoders Music2Latent, Stable Audio VAE, SD VAE; a 410M-parameter DiT-style Transformer with AdamW, log-normal λ sampling, early stopping.

Gaussianity checks: audio configurations satisfy near-Gaussian statistics (skewness/kurtosis/correlation deviations small); CelebA + SD VAE violates them, yet the bell shape persists with a shifted peak.
MAESTRO normalized train–test gap: ~0.02 at boundaries (λ∈{0,1}), peaking near 0.09 at λ≈0.5–0.6, matching theory.
Ablations: predicted $\lambda^*_F$ matches observed peaks across data distributions (MAESTRO 0.52 vs. 0.5–0.6; MTG-Jamendo 0.37 vs. 0.3–0.4; FMA 0.42 vs. 0.4–0.5) and noise scalings (×0.25→0.31, ×1→0.52, ×4→0.59).
Architecture/capacity/scheduler affect magnitude, not location: Transformer 0.09 vs. UNet 0.01 (≈9× stronger); 140M→410M→880M gives 0.06→0.09→0.12; log-normal scheduler 0.09 vs. uniform 0.06.
Linear/nonlinear competition: the Transformer-to-OLS test-loss ratio peaks exactly at $\lambda^*$ , confirming that nonlinear capacity drives the leakage.

Membership inference attack

The attack: sample K=100 noise vectors, evaluate the model's one-step reconstruction MSE over 11 λ values (0 to 1.0), and feed the 11-dimensional λ-profile to an MLP classifier. Requires only forward passes — no gradients or weights.

| Method | AUC | |---|---| | Naive (single λ*) | 0.67 | | SecMI (adapted) | 0.72 | | PIA (adapted) | 0.83 | | λ-profile MLP | 0.91 |

Exploiting the full bell-shaped structure clearly beats single-point attacks.

Defenses and tradeoffs

$\lambda^*_F$ is architecture-independent, so attackers can locate it on small proxy models and transfer it.
Defenders should concentrate privacy mechanisms (e.g., differential privacy, regularization) near $\lambda^*$ rather than uniformly along the path.
One reflow step preserves the bell shape but greatly attenuates its amplitude — a potential built-in mitigation.
Efficiency–privacy tradeoff: Esser et al. (2024) found that concentrating $p(\lambda)$ near 0.5 improves SD3 convergence, but this is exactly where membership leakage peaks — faster training may amplify the privacy hole.

Limitations

The closed-form $\lambda^*_F$ requires near-Gaussian isotropic latents; CelebA + SD VAE breaks the prediction (predicted 0.45, observed 0.6–0.7) though the bell shape persists.
Independent-coupling assumption excludes reflow (preliminary results show attenuation).
White-box, unconditional-generation setting only; conditional generation would modify $\Sigma_1$ and hence $\lambda^*_F$ .
Experiments capped at 880M parameters — behavior at FLUX/SD3 scale (billions of parameters) remains open.

Takeaways

1. Membership signals follow a universal bell-shaped curve over λ with a geometry-predicted peak. 2. Larger models amplify the signal; larger datasets attenuate it. 3. Standard metrics cannot detect the leakage while it silently accumulates. 4. A leak map is worth more than a hundred attacks: know where your $\lambda^*$ is, and harden precisely there.

Reference: Sesmat et al., *Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path*, arXiv:2606.07271.