Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning

Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing

Overview

Standard semi-supervised learning (SSL) treats all unlabeled data equally by using a single scalar weight to balance supervised and unsupervised losses. However, forcing a model to learn from ambiguous or incorrectly pseudo-labeled data can actively harm performance. This paper introduces a method to assign a unique, learnable weight to every unlabeled example using influence functions. By dynamically down-weighting unhelpful data, the approach achieves state-of-the-art results on both image and text classification benchmarks. To make this computationally tractable, the authors propose clever approximations for per-example gradients and inverse Hessians.

The Challenge of Equal Weighting in SSL

In SSL, models are trained on a small set of labeled data and a massive set of unlabeled data. Modern algorithms like UDA or FixMatch rely on consistency regularization: generating a "pseudo-label" for an unlabeled example and forcing the model to predict that same label even if the input is heavily augmented.

Typically, the overall loss function combines supervised and unsupervised loss, balanced by a single hyperparameter $\lambda$. But not all unlabeled data is equally useful. If the model generates an incorrect pseudo-label for an ambiguous image, learning from it degrades the model. Manually tuning a separate weight for thousands of individual unlabeled examples is impossible. The authors tackle this by automatically learning these per-example weights to maximize performance on a small, trusted validation set.

A Bi-Level Optimization Approach

The authors formulate this as a bi-level optimization problem: update the model parameters to minimize training loss, and update the per-example unlabeled weights to minimize validation loss.

To determine how changing a specific training example's weight affects validation loss, they use influence functions. Because computing influence functions for deep neural networks is notoriously slow and memory-intensive, they introduce three key technical innovations:

  1. Efficient Per-Example Gradients: Standard backpropagation aggregates gradients across a batch. To get gradients for individual examples without a slow serial loop, they use auto-differentiation tools to grab the gradient with respect to a layer's activations (stored per-example), then apply the chain rule to get parameter gradients via simple element-wise multiplication.
  2. Last-Layer Hessian Approximation: Influence functions require computing an inverse Hessian matrix, which is impossibly large for modern networks. The authors approximate this by only computing the exact inverse Hessian for the last layer (the classifier), assuming earlier feature-extraction layers are frozen during the influence calculation.
  3. Masked-Adam Optimizer: Because they sample a mini-batch of unlabeled data at a time, they only update a sparse subset of per-example weights at each step. Standard optimizers (like Adam) incorrectly update running momentum statistics for the entire dataset. The authors introduce "M-Adam" to selectively update statistics only for the sampled batch.

Key Insights and Visualizations

The Model Learns to Ignore Its Own Mistakes

On synthetic datasets, visualizations clearly show the algorithm actively down-weighting unlabeled examples that sit on the "wrong" side of the learned decision boundary. By ignoring these confusing points, the decision boundary smoothly converges to the ground truth.

Evolution of decision boundaries and data weighting on Circles and Moons datasets

The Last Layer is All You Need (for Influence)

Approximating the influence function using only the last layer's parameters isn't just a memory-saving hack; it performs exceptionally well. The authors suspect that because earlier layers converge quickly, the last layer's exact inverse Hessian provides a highly accurate signal for the influence function in later training stages.

Optimizer State Matters for Sparse Updates

Correctly handling momentum for sparse hyperparameter updates is critical. As shown in the ablation study, the custom Masked-Adam (M-Adam) optimizer significantly outperforms vanilla Adam and basic SGD in this specific bi-level setup.

Validation error comparison showing M-Adam outperforming SGD and standard Adam

Performance and Efficiency Results

The per-example weighting scheme improves upon strong state-of-the-art baselines across multiple domains:

  • Accuracy: On CIFAR-10 (4000 labels), it achieves a 4.35% error rate, beating the FixMatch baseline (4.48%). On IMDb text classification (with only 20 labeled examples), it achieves an 8.51% error rate, outperforming the UDA baseline (8.98%).
  • Speed: The proposed per-example gradient computation is 19.3x faster than a standard serial implementation (0.94 seconds vs. 18.17 seconds per batch).
  • Memory Efficiency: Restricting the Hessian computation to the last layer keeps the GPU memory footprint to just 9GB, compared to 15.8GB+ for competing approximation methods.

Limitations

  • Hardware Constraints: Text classification experiments (IMDb) were limited to a maximum sequence length of 128 tokens due to 16GB GPU memory limits, whereas standard UDA achieves its best results using sequence lengths of 512 on larger hardware.
  • Validation Set Reliance: Like all influence-function-based methods, this approach requires a trusted validation set. The quality of the learned weights is inherently bottlenecked by the representativeness of this validation data.

Summary updated

Abstract

Existing semi-supervised learning (SSL) algorithms use a single weight to balance the loss of labeled and unlabeled examples, i.e., all unlabeled examples are equally weighted. But not all unlabeled data are equal. In this paper we study how to use a different weight for “every” unlabeled example. Manual tuning of all those weights -- as done in prior work -- is no longer possible. Instead, we adjust those weights via an algorithm based on the influence function, a measure of a model's dependency on one training example. To make the approach efficient, we propose a fast and effective approximation of the influence function. We demonstrate that this technique outperforms state-of-the-art methods on semi-supervised image and language classification tasks.
BibTeX
@inproceedings{ren2020not,
  author = {Zhongzheng Ren and Raymond A. Yeh and Alexander G. Schwing},
  title = {{Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=8_BRqSXcgNr},
}
Figure