MIND: Decoupling Model-Induced Label Noise from the Feature Manifold (ICML 2026)

Forum topic · 小凯 · 2026-05-18

Summary

A forum post discusses MIND (arXiv:2605.16081), an ICML 2026 paper by Ren addressing a systematic bias in using pretrained models as annotators. Unlike random label noise, model-induced noise is structural: it is coupled to local regions of the feature manifold, so global confusion matrices cannot describe it, while per-sample noise matrices are mathematically ill-posed. MIND performs latent disentanglement, decomposing the high-dimensional noise manifold into low-dimensional subspace components via a Latent Disentanglement Estimator (LDE), which dynamically projects samples into latent clusters with consistent error patterns, restoring noise identifiability. Evaluation spans controlled noise on CIFAR-100 and large-scale 3D datasets (S3DIS, ScanNet) with geometry-driven structured noise, where MIND significantly outperforms prior state of the art. It can also correct zero-shot hallucinations from vision-language models such as OpenSeg. The post raises open questions on cluster-count selection, robustness to per-sample noise, and scalability.

Overview

A common practice in deep learning is to use pretrained or foundation models to auto-generate labels, replacing costly human annotation. But if the annotator model itself is biased, the generated labels carry *systematic* errors — not random ones. These are the model's inductive biases: it consistently misjudges certain feature patterns and is consistently overconfident about others.

MIND (*Decoupling Model-Induced Label Noise via Latent Manifold Disentanglement*, Ren, ICML 2026) distinguishes two kinds of noise:

Random noise: independent of the feature distribution, like white noise — the target of classic robust learning methods.
Model-induced noise: structural and tightly coupled to the local feature manifold. All samples in one region of feature space may be corrupted by the same misjudgment pattern. A global confusion matrix cannot describe this, yet learning an independent noise matrix per sample is mathematically ill-posed.

Key points

Latent disentanglement: MIND decomposes the high-dimensional noise manifold into multiple low-dimensional subspace components.
Latent Disentanglement Estimator (LDE): dynamically projects samples into latent structural clusters, each with a consistent error pattern. Once samples are grouped, noise identifiability is restored.
Hierarchical evaluation: benchmarks on CIFAR-100 with controlled noise, then transfer to large-scale real-world 3D datasets (S3DIS, ScanNet), where error patterns are driven by geometric feature manifolds rather than added randomly. On such structured-noise scenarios, MIND significantly outperforms existing state-of-the-art methods.
Bonus result: MIND can correct zero-shot hallucinations from vision-language models such as OpenSeg.

Open questions (from the post)

Is the number of LDE clusters preset or automatically determined?
If real error patterns are extremely complex (e.g., unique noise per sample), does the latent disentanglement assumption still hold?
Computational cost: LDE clusters dynamically during training — how does this scale to very large datasets?

References

1. Ren, D. (2026). *MIND: Decoupling Model-Induced Label Noise via Latent Manifold Disentanglement*. arXiv:2605.16081 [cs.LG]. 2. Han, B., et al. (2018). *Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels*. NeurIPS. 3. Zhang, Y., et al. (2023). *Label-Error Detection in Language Model Data*. ICML.