Global Pseudo Labeling via Optimal Transport

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2281 papers384 conceptsfrozen 2026-10-04

In semi-supervised and unsupervised learning, the model must often act as its own teacher. The standard way to do this is pseudo-labeling: the model takes an unlabeled image, predicts a class, and—if it is confident enough—treats that prediction as a ground-truth label for future training.

However, this "greedy" approach has a fatal flaw. If a model is slightly biased toward a majority class, it will produce more pseudo-labels for that class, which in turn makes the model even more biased. This feedback loop leads to model collapse, where the network eventually predicts the same few "easy" classes for everything.

To break this loop, a growing body of research reframes pseudo-labeling not as a series of independent local decisions, but as a global resource allocation problem solved through Optimal Transport (OT).

From Greedy Thresholds to Global Matching

Traditional pseudo-labeling, such as the approach used in FixMatch, relies on a confidence threshold. If the model's output for a sample is above 0.95, it gets a label; otherwise, it is ignored. As Hisham Husain and colleagues point out in Confident Sinkhorn Allocation (CSA), this ignores the global structure of the data.

By contrast, the Optimal Transport framework treats the unlabeled samples and the available class labels as two distributions that need to be matched. Imagine you have a batch of 100 samples and you know (or estimate) that your dataset should have an equal number of cats and dogs. Instead of letting the model pick "cat" for 90 samples because it finds them "easier," OT forces the model to find the 50 best "cats" and the 50 best "dogs" to satisfy the global distribution.

As Kai Sheng Tai demonstrates in Sinkhorn Label Allocation (SLA), this is mathematically formulated as finding a transport plan that minimizes the cost of moving "mass" from samples to classes. The "cost" is typically the negative log-probability predicted by the model.

The Sinkhorn-Knopp Engine

The practical engine behind this shift is the Sinkhorn-Knopp algorithm. Solving a standard Optimal Transport problem is computationally expensive, but by adding an entropic regularization term, the problem becomes solvable through simple, iterative row and column normalization of the prediction matrix.

This mechanism allows models to enforce class balance dynamically. In FedPDM, Dongping Liao uses this to prevent local models in federated learning from becoming overconfident in biased local data. Similarly, Wanxing Chang uses it in CSOT to map samples to class centroids, ensuring that the resulting pseudo-labels reflect expected proportions rather than model whims.

Neutralizing the Long-Tail and Noise

The "global" nature of OT is particularly powerful when dealing with real-world "messy" data—specifically long-tailed distributions (where some classes are rare) and label noise.

In SoLar, Haobo Wang shows that by setting a prior distribution that accounts for the "tail" classes, OT prevents the model from simply ignoring rare categories. He Zhao extends this logic in Extracting Clean and Balanced Subsets, using OT to simultaneously filter out mislabeled samples and correct class imbalance. Because OT considers the entire batch, a noisy sample that might look "confident" in isolation is often rejected because it doesn't fit the global geometric structure of the class it claims to belong to.

Training-Free Adaptation

Perhaps the most surprising application of this idea is that it doesn't always require retraining. Zero-shot models like CLIP often suffer from "label distribution bias," where they over-predict classes that were common in their massive pretraining sets.

Changho Shin and Jitian Zhao introduced OTTER, a method that applies OT at inference time. By treating the classification of a test batch as a transport problem, OTTER re-aligns the model's raw scores to match the target task's distribution. This simple "wrapper" can boost accuracy by over 15% on text tasks without a single gradient update, proving that the problem with many models isn't their underlying features, but how they aggregate those features into final decisions.

Why It Works: Geometric Awareness

The core intuition is that Optimal Transport introduces geometric awareness. While a standard softmax classifier treats every sample as an island, OT-based pseudo-labeling treats the dataset as a landscape. By enforcing constraints on the "marginal" distributions (the total counts of each class), researchers like Chuang Zhu in ProML ensure that the model's internal representation of a class remains distinct and balanced, even when labeled data is scarce.

Papers in this version

Showing 60 of 281 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 221 papers
  2. Version 1 · 2026-10-04 · 281 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_optimal_transport_pseudo_labeling_a4b5f1f5,
  title = {optimal transport pseudo-labeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/optimal-transport-pseudo-labeling/rdr_a4b5f1f5e42a4783a70457816b8bb321},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_global_pseudo_labeling_via_optimal_trans_a4b5f1f5_v1,
  title = {Global Pseudo Labeling via Optimal Transport},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/global-pseudo-labeling-via-optimal-transport/rdr_a4b5f1f5e42a4783a70457816b8bb321/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}