Correcting Label Corruption with Noise Transition Matrices

This direction is merged into another direction: noise transition matrix estimation. Its last version stays readable and citable here.

Version 1 of 175 papers80 conceptsfrozen 2026-10-04

Deep neural networks are remarkably effective at pattern recognition, but they possess a dangerous "superpower": they can memorize almost anything. When a dataset contains incorrect labels—whether from human error, automated scraping, or sensor noise—a standard network will eventually treat these errors as ground truth, leading to poor generalization.

The research direction of Learning with Noisy Labels (LNL) seeks to build models that can see through this corruption. The central mechanism in this field is the Noise Transition Matrix, a mathematical bridge that describes how "clean" labels are systematically flipped into "noisy" ones.

The Blurring Process

To understand why this works, we can think of label noise as a blurring operation. If $y$ is the true, hidden label and $\tilde{y}$ is the noisy label we actually observe, the relationship between them can be modeled by a transition matrix $T$. Each entry $T_{ij} = P(\tilde{y}=j | y=i)$ represents the probability that an instance belonging to class $i$ is incorrectly labeled as class $j$.

In their seminal work, Giorgio Patrini, Alessandro Rozza, and Aditya Krishna Menon established that if we can estimate this matrix $T$, we can "unblur" the training process using Loss Correction.

Forward and Backward Correction

There are two primary ways to use the transition matrix to achieve statistical consistency—meaning the model behaves as if it were trained on perfectly clean data.

  1. Backward Correction: This method attempts to reconstruct the "true" loss. By multiplying the observed loss by the inverse of the transition matrix ($T^{-1}$), we can theoretically recover the loss relative to the hidden clean labels. However, as noted by Kiran Koshy Thekumparampil and Sewoong Oh in Robustness of conditional GANs to noisy labels, this is often numerically unstable because small errors in $T$ are amplified during inversion.
  2. Forward Correction: Instead of unblurring the loss, we "blur" the model's predictions. We take the model's estimate of the clean probability and multiply it by $T$ to produce a "noisy" prediction. We then compare this noisy prediction to the noisy labels. This is generally more stable and prevents the model from overfitting to the noise.

The Estimation Bottleneck

The effectiveness of these methods hinges entirely on the accuracy of $T$. Traditionally, researchers relied on Anchor Points—data samples that belong to a specific class with 100% certainty. By observing how these "perfect" samples are labeled in the noisy dataset, one can directly read off the transition probabilities.

However, Tongliang Liu and Xiaobo Xia argue in Are Anchor Points Really Indispensable in Label-Noise Learning? that such points are rare in real-world data. They introduced T-Revision, which treats the transition matrix as a learnable parameter that can be refined during training, relaxing the need for perfect anchor points.

To further reduce estimation error, Tongliang Liu and Bo Han proposed the Dual-T estimator in Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning. This approach factorizes the complex transition matrix into two simpler, intermediate matrices, significantly improving the stability of the estimation.

Filtering and Peer Learning

While transition matrices provide a global statistical fix, other researchers focus on sample-level filtering. The intuition here is the "Small Loss" hypothesis: DNNs tend to learn simple, clean patterns before they start memorizing complex noise.

In Co-teaching: Robust training of deep neural networks with extremely noisy labels, Bo Han and Gang Niu introduced a peer-filtering mechanism. Two networks are trained simultaneously; each network identifies "clean" (low-loss) samples and feeds them to its peer. This "cross-pollination" prevents a single network from confirming its own biases about which samples are noisy.

Alternatively, Yang Liu and Hongyi Guo proposed Peer Loss Functions in Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates. By comparing a model's prediction against a random "peer" label from the dataset, the bias introduced by label noise is implicitly cancelled out, removing the need to estimate $T$ entirely.

From Class-Conditional to Instance-Dependent Noise

Most early work assumed that the probability of a label flip depends only on the class (e.g., a "cat" is always 10% likely to be labeled a "dog"). However, real-world noise is often Instance-Dependent (IDN)—a blurry photo of a cat is more likely to be mislabeled than a clear one.

Recent frameworks like ReIDVAE, developed by Hanwen Deng and Min-Ling Zhang in Rethinking the Value of Labels for Instance-Dependent Label Noise Learning, use generative modeling to disentangle "content" from "noise style." This allows the model to identify clean labels even when the noise patterns are inextricably linked to the specific features of the image.

Papers in this version

Showing 60 of 75 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_correcting_label_corruption_with_noise_t_9d74c19c,
  title = {Correcting Label Corruption with Noise Transition Matrices},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/correcting-label-corruption-with-noise-transition-matrices/rdr_9d74c19ce402415a98b4d076ebb13bd7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_correcting_label_corruption_with_noise_t_9d74c19c_v1,
  title = {Correcting Label Corruption with Noise Transition Matrices},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/correcting-label-corruption-with-noise-transition-matrices/rdr_9d74c19ce402415a98b4d076ebb13bd7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}