Geometric Identifiability for Label Noise Learning

This direction is merged into another direction: instance-dependent label noise. Its last version stays readable and citable here.

Version 1 of 115 papers28 conceptsfrozen 2026-10-04

In real-world machine learning, we rarely have access to "ground truth" labels. Instead, we deal with "noisy" labels—annotations from tired crowdsourced workers, automated web scrapers, or non-expert users in federated systems. Traditionally, to recover the true signal from this noise, researchers assumed the existence of "anchor points": specific data samples that belong to a class with 100% certainty.

However, as Xuefeng Li, Tongliang Liu, and Bo Han argue in Provably End-to-end Label-noise Learning without Anchor Points, anchor points are rare in the wild. A new family of methods has emerged that bypasses this requirement by treating label noise as a geometric problem in high-dimensional probability space.

The Geometry of Corruption

To understand this approach, imagine the space of all possible class probabilities as a simplex—a geometric shape where each vertex represents a "pure" class (e.g., 100% Dog, 0% Cat).

When labels are clean, the model's predictions should ideally sit at these vertices. However, label noise acts as a mathematical contraction. As Yivan Zhang, Gang Niu, and Masashi Sugiyama explain in Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization, noise "blurs" the distributions, pulling the observed noisy labels away from the vertices and toward the center of the simplex. This reduces the statistical distance (or "contrast") between classes, making them harder to distinguish.

The core insight of this research direction is that we can recover the clean signal by finding the geometric transformation that "undoes" this contraction.

Simplex Volume Minimization

If we assume that the true class posteriors are sufficiently scattered, the noisy labels will form a smaller shape inside the original simplex. The Noise Transition Matrix ($T$) is the operator that maps clean labels to noisy ones.

In VolMinNet, the researchers show that the true transition matrix is the one that forms the minimum volume simplex enclosing all observed noisy posterior probabilities. By minimizing this volume, the model is forced to find the "tightest" possible fit for the noise, effectively pushing the estimated clean labels back toward the vertices of the probability space.

This principle has been extended to complex scenarios like domain adaptive semantic segmentation. In SimT, Xiaoqing Guo and Yixuan Yuan use simplex volume regularization to handle "mixed noise"—where a model must deal with both mislabeled known classes and entirely unknown "open-set" categories.

Total Variation and Contrast Maximization

While volume minimization focuses on the transition matrix, another perspective focuses on the predictions themselves. Since noise is a contraction that reduces contrast, we can recover the clean signal by seeking the distribution that maximizes contrast.

Yivan Zhang and colleagues propose using Total Variation (TV) Regularization to maximize the pairwise distance between predicted class probabilities. By encouraging the model to produce distinct, high-contrast predictions, they effectively reverse the blurring effect of the noise. This approach identifies the underlying clean labels among many mathematically valid possibilities by choosing the one that is "most distinguishable."

Robustness in Decentralized and Crowdsourced Settings

These geometric principles are particularly powerful when data is scarce or experts are unavailable:

  • Crowdsourcing: In Deep Learning From Crowdsourced Labels, Shahana Ibrahim and Xiao Fu introduce GeoCrowdNet. It uses volume maximization to distinguish true signals from annotator confusion, ensuring accuracy even when no "expert" annotators exist in the pool.
  • Federated Learning: Xiaochen Zhou and Xudong Wang apply similar geometric constraints in Federated Label-Noise Learning. By enforcing the parsimony of the noise transition matrix locally, they allow individual clients to learn effectively even with very few samples.
  • Clustering: In VolMaxDCC, Tri Nguyen uses volume maximization to prevent noisy pairwise constraints from creating "uncertain" cluster assignments, pushing the model toward sharp, distinct memberships.

By shifting the problem from "finding clean examples" to "optimizing the geometry of the probability space," these researchers have created a framework for learning that is provably robust, even when the data is heavily corrupted and no ground truth is in sight.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_geometric_identifiability_for_label_nois_399e8caa,
  title = {Geometric Identifiability for Label Noise Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometric-identifiability-for-label-noise-learning/rdr_399e8caa39c746eba4b300e6888aecc6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_geometric_identifiability_for_label_nois_399e8caa_v1,
  title = {Geometric Identifiability for Label Noise Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometric-identifiability-for-label-noise-learning/rdr_399e8caa39c746eba4b300e6888aecc6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}