Learning from What an Input Is Not

499 concepts217 papers1 proposal

In semi-supervised learning, unsupervised domain adaptation, and noisy-label classification, models face a fundamental dilemma: how to learn from data without reliable ground-truth labels. The conventional approach, known as positive pseudo-labeling, forces a model to guess the single most likely class for an unlabeled or ambiguous sample and treat that guess as absolute truth.

However, deep neural networks are notoriously overconfident in their mistakes. When a model is forced to make a positive prediction on a highly uncertain or out-of-distribution sample, it introduces noise. During self-training, this noise loops back into the optimization process, causing confirmation bias, error propagation, and a complete collapse of decision boundaries. To avoid this, traditional frameworks simply discard low-confidence samples. But throwing away data means losing valuable structural information about the underlying data distribution.

The Statistical Safety of Exclusion

What if, instead of asking what a sample is, we ask what it definitely is not? This shift from positive learning to negative learning forms the core of a powerful paradigm designed to handle uncertainty and noise.

As surveyed by Guillaume Sanchez et al. in Deep Learning Classification With Noisy Labels, providing a model with complementary labels—specifying incorrect classes—is inherently more robust in high-noise environments. The intuition is fundamentally statistical. In a multi-class classification problem with $K$ classes, guessing the single true label is risky; if the data is corrupted, the chance of picking the wrong label is high.

Conversely, as demonstrated by Youngdong Kim et al. in NLNL: Negative Learning for Noisy Labels, the probability that a randomly selected negative label accidentally conflicts with the true class is only $1/(K-1)$. As the number of classes increases, the negative signal becomes mathematically safer, preventing the model from memorizing incorrect positive associations.

Salvaging Low-Confidence Data

This statistical safety allows models to salvage the vast amounts of low-confidence data that traditional self-training pipelines discard. When a model evaluates an ambiguous sample, its top prediction might be highly uncertain. However, its lowest-probability predictions are often highly certain; the model is absolutely sure the sample does not belong to those classes.

In the context of unsupervised domain adaptation, Jaemin Na et al. show in FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation that these low-confidence predictions can be repurposed as negative pseudo-labels to penalize unlikely class assignments. Instead of ignoring ambiguous target-domain samples, the model explicitly learns what they are not, which refines decision boundaries in the new domain.

Similarly, for test-time adaptation, Adilbek Karmanov et al. demonstrate in Efficient Test-Time Adaptation of Vision-Language Models that storing high-confidence negative predictions in a dynamic cache prevents common misclassifications in out-of-distribution scenarios. By modeling what an image is definitely not, the system filters out noise and stabilizes inference without requiring expensive backpropagation.

Carving Decision Boundaries via Contrastive Exclusion

How exactly does a model mathematically operationalize a negative pseudo-label? The most common mechanism is integrating negative labels into contrastive learning frameworks or specialized loss functions that push feature representations away from incorrect class centers.

The authors of In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning introduce a specialized loss that penalizes the model if it assigns a high probability to classes identified as highly unlikely. This acts as a soft constraint on the decision boundaries during the early stages of training when positive pseudo-labels are still highly unreliable.

To make this representation space even cleaner, many recent approaches combine negative labeling with contrastive loss functions. For instance, Qinyi Deng et al. propose a framework in Boosting Semi-Supervised Learning with Contrastive Complementary Labeling where negative pairs are explicitly formed by matching a low-confidence sample with high-confidence samples from its identified unlikely classes. This contrastive signal pushes the representation of the ambiguous sample away from the regions of the feature space dedicated to those incorrect categories.

This mechanism is highly effective across diverse modalities:

A Safety Mechanism Against Noise and Open-Set Risks

Beyond salvaging data, negative learning acts as a critical safety mechanism that prevents error propagation. In self-training pipelines, a single incorrect positive pseudo-label can permanently skew a model's parameters. By contrast, negative learning provides a much gentler gradient.

Yunan Liu et al. leverage this in Hierarchical Information Passing Based Noise-Tolerant Hybrid Learning for Semi-Supervised Human Parsing, treating uncertain pseudo-labels with negative learning to extract useful gradients without the risk of overfitting to specific errors. Instead of forcing convergence on a dubious label, the network simply learns what to avoid. In a similar vein, Tao Tong argues in Meta Label Correction with Generalization Regularizer that formulating noisy labels as negative signals that the model must actively avoid predicting is far more effective for robust training than simply down-weighting or ignoring suspicious samples.

This exclusionary logic also solves a notorious flaw in active learning within open-set environments: the tendency of models to select uninformative, out-of-distribution samples because they look "uncertain." As shown by Chen-Chen Zong et al. in Bidirectional Uncertainty-Based Active Learning for Open-Set Annotation, applying negative learning with random labels to these highly uncertain samples pushes unknown-class instances into high-confidence regions. This effectively isolates them from the informative, known-class samples that remain in low-confidence zones, allowing the active learning algorithm to focus exclusively on data points that genuinely benefit the model.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 242 papers
  2. Version 1 · 2026-10-04 · 344 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_complementary_label_learning_37600e96,
  title = {complementary label learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/complementary-label-learning/rdr_37600e9641b24325872ff1640456652f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 2, frozen text):

@misc{lacuna_complementary_label_learning_37600e96_v2,
  title = {complementary label learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/complementary-label-learning/rdr_37600e9641b24325872ff1640456652f/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake