Negative Learning and Contrastive Constraints for Unlabeled Data

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2344 papers580 conceptsfrozen 2026-10-04

In semi-supervised learning, the standard recipe for handling unlabeled data is simple: if the model is confident about a prediction, treat that prediction as a "pseudo-label" and train on it. If the model is not confident, throw the data away.

While effective, this thresholding strategy is inherently wasteful. In many datasets, a vast majority of samples never reach the high confidence required (often 95% or higher) to be included in training. Researchers like Yuchao Wang and Haochen Wang argue in Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels that "every pixel matters." They suggest that even when a model doesn't know what a sample is, it almost certainly knows what it is not.

This insight has sparked a research direction focused on recycling low-confidence data. Instead of discarding uncertain samples, these methods use them as negative constraints—either through "Negative Learning" (NL) or as negative anchors in contrastive frameworks.

The Intuition of Exclusion

To understand why this works, consider a classification task with 100 classes. If a model looks at a blurry image and assigns a 20% probability to "Dog," a standard pipeline discards it. However, the model might assign a 0.001% probability to "Ship," "Airplane," and "Car."

While the model is too uncertain to claim the image is a dog, it is highly certain the image is not a ship. Youngdong Kim and colleagues formalized this in NLNL: Negative Learning for Noisy Labels. By training the model to minimize the probability of these "complementary labels," we provide a supervised signal that is much less likely to be wrong than a guessed positive label. This prevents the "confirmation bias" where a model drifts by repeatedly training on its own incorrect guesses.

Recycling via Contrastive Learning

A second major mechanism for using uncertain data is through feature-level contrastive learning. In this framing, low-confidence samples are not given a label at all. Instead, they are used to shape the underlying feature space.

In the U2PL framework proposed by Haochen Wang, high-confidence pixels act as positive anchors, while low-confidence pixels are used as negative samples. By pushing the representations of these "unreliable" pixels away from the prototypes of classes they likely don't belong to, the model learns sharper, more discriminative decision boundaries.

Similar logic is applied in Ada-CM by Hangyu Li, where contrastive learning on low-confidence samples allows the model to capture structural information and robust features that would otherwise be lost. This is particularly useful in domains like facial expression recognition or medical imaging (Jinxi Xiang), where visual ambiguity is high.

Bridging Domains and Environments

This "exclusionary" approach is especially powerful when the data distribution shifts. In Unsupervised Domain Adaptation (UDA), the gap between a labeled source (e.g., daytime images) and an unlabeled target (e.g., nighttime images) is often too large for simple pseudo-labeling.

Jaemin Na and Wonjun Hwang demonstrated in FixBi that using low-confidence target predictions as negative pseudo-labels helps penalize unlikely class assignments. This "self-penalization" refines the model's understanding of the target domain without requiring the model to be "right" about the positive class early in training.

In decentralized settings, Jieming Bian introduced FedSEAL, which uses negative learning to utilize uncertain data across federated clients. This provides a "safer" gradient signal in the early stages of training, ensuring that decentralized models don't converge on noise before they've had a chance to see enough diverse data.

Why It Works: Breaking the Feedback Loop

The fundamental advantage of these methods is the mitigation of error accumulation.

  1. Safety: In a 10-class problem, a random positive guess has a 90% chance of being wrong. A random negative guess (selecting one class it is not) has only a 10% chance of being wrong.
  2. Boundary Calibration: High-entropy (uncertain) samples usually lie near the decision boundaries. By using them as negative constraints, as seen in the work of Xinyu Chen on vision-language models, we explicitly "push" the boundaries away from these ambiguous regions, leading to better-calibrated classifiers.
  3. Information Density: Even "noisy" or "unreliable" data contains structural information. As Yanshuo Wang shows in Dynamic Sample Selection, extracting "what it is not" allows models to maintain accuracy in high-uncertainty scenarios like continual test-time adaptation.

By shifting the focus from "what is this?" to "what could this not be?", this research direction transforms previously discarded noise into a robust signal for representation learning.

Papers in this version

Showing 60 of 344 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 242 papers
  2. Version 1 · 2026-10-04 · 344 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_complementary_label_learning_37600e96,
  title = {complementary label learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/complementary-label-learning/rdr_37600e9641b24325872ff1640456652f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_negative_learning_and_contrastive_constr_37600e96_v1,
  title = {Negative Learning and Contrastive Constraints for Unlabeled Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/negative-learning-and-contrastive-constraints-for-unlabeled-data/rdr_37600e9641b24325872ff1640456652f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}