MIND: Decoupling Model-Induced Label Noise via Latent Manifold Disentanglement

Forum topic · 小凯 · 2026-05-18

Summary

Forum post discusses MIND, an ICML 2026 paper by Ren addressing a blind spot in deep learning: using pretrained or foundation models as automatic annotators introduces systematic, model-induced label noise rather than random noise. Unlike classic robust-learning settings where noise is feature-independent (like white noise), model-induced noise is structurally coupled to the local feature manifold—an entire region of feature space may be corrupted by the same misjudgment pattern, making a global confusion matrix insufficient while per-sample noise matrices are ill-posed. MIND's solution is latent disentanglement: it decomposes the high-dimensional noise manifold into low-dimensional subspace components via a Latent Disentanglement Estimator (LDE) that dynamically projects samples into latent structural clusters where error patterns are consistent, restoring noise identifiability. Evaluation spans CIFAR-100 with controlled noise and large-scale real-world 3D datasets (S3DIS, ScanNet), where error modes are driven by geometric feature manifolds. MIND significantly outperforms state-of-the-art methods under such structured noise and can also correct zero-shot hallucinations from vision-language models like OpenSeg. Open questions include how LDE cluster counts are determined, robustness of the latent disentanglement assumption under highly complex per-sample noise, and scalability of dynamic clustering on large datasets.

Deep learning has a common practice: use pretrained models or foundation models to automatically generate labels, replacing expensive human annotation. But have you considered—if the annotator itself is biased, then the labeled data contains systematic errors?

This is not random noise—not a cat picture mislabeled as a dog. It is the inductive bias of the pretrained model: it consistently misjudges certain feature patterns while being overconfident about others.

Two Kinds of Noise

Ren's ICML 2026 paper distinguishes two types of noise. Classical robust learning handles *random noise*—independent of the feature distribution, uniform like white noise. But *model-induced noise* is structural—it is tightly coupled to the local feature manifold. All samples in a region of feature space may be corrupted by the same misjudgment pattern. A global confusion matrix is insufficient to describe this noise, because the error pattern differs per local region. Yet learning an independent noise matrix per sample is mathematically ill-posed.

MIND's Approach: Latent Disentanglement

MIND's solution is latent disentanglement: decompose the high-dimensional noise manifold into multiple low-dimensional subspace components. The core component is the Latent Disentanglement Estimator (LDE), which dynamically projects samples into latent structural clusters—within each cluster, the error pattern is consistent. Once samples are grouped, identifiability of the noise is restored.

Evaluation

The evaluation is layered: first a benchmark on CIFAR-100 with controlled noise, then transfer to large-scale real-world 3D datasets (S3DIS, ScanNet)—where the error patterns are driven by the geometric feature manifold, not randomly injected. In these structured-noise scenarios, MIND significantly outperforms existing state-of-the-art methods.

An impressive additional result: MIND can correct zero-shot hallucinations from vision-language models (e.g., OpenSeg).

Open Questions

Is the number of LDE clusters preset or automatically determined?
If the true error patterns are highly complex (e.g., every sample has a unique noise pattern), does the latent disentanglement assumption still hold?
Computational cost: LDE requires dynamic clustering during training—how does it scale to large datasets?

---

References

1. Ren, D. (2026). *MIND: Decoupling Model-Induced Label Noise via Latent Manifold Disentanglement*. arXiv:2605.16081 [cs.LG]. 2. Han, B., et al. (2018). *Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels*. NeurIPS. 3. Zhang, Y., et al. (2023). *Label-Error Detection in Language Model Data*. ICML.