Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning

Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing

Abstract

Existing semi-supervised learning (SSL) algorithms use a single weight to balance the loss of labeled and unlabeled examples, i.e., all unlabeled examples are equally weighted. But not all unlabeled data are equal. In this paper we study how to use a different weight for every unlabeled example. Manual tuning of all those weights -- as done in prior work -- is no longer possible. Instead, we adjust those weights via an algorithm based on the influence function, a measure of a model's dependency on one training example. To make the approach efficient, we propose a fast and effective approximation of the influence function. We demonstrate that this technique outperforms state-of-the-art methods on semi-supervised image and language classification tasks.

Introduction: The Challenge of Uniform Weighting in SSL

In semi-supervised learning (SSL), models are typically trained on a small set of labeled data and a significantly larger pool of unlabeled data. Modern SSL frameworks, such as FixMatch or Unsupervised Data Augmentation (UDA), rely on generating "pseudo-labels" for unlabeled points to guide training.

A fundamental limitation of these approaches is the use of a single global weight ($\lambda$) to balance the contribution of labeled versus unlabeled data. This uniform weighting assumes all unlabeled samples are equally reliable. However, incorrect pseudo-labels can introduce noise that sabotages model performance. Ideally, a model should down-weight ambiguous or noisy unlabeled samples while up-weighting those that provide clear, helpful signals. Manually tuning individual weights for millions of samples is impossible; this paper introduces an automated method to learn a unique weight for every unlabeled example.

Methodology: Bi-Level Optimization and Influence Functions

The authors frame the weighting problem as a bi-level optimization problem. The "inner loop" trains the model parameters ($\theta$) to minimize the weighted training loss, while the "outer loop" adjusts the per-example weights ($\Lambda$) to minimize loss on a separate validation set.

To solve this efficiently, the researchers utilize Influence Functions, which quantify how much a model's parameters would shift if a specific training point were slightly up-weighted. To make this scalable for deep neural networks, the paper introduces three technical innovations:

  1. Fast Per-example Gradients: An efficient extension of backpropagation that avoids the serial processing bottleneck.
  2. Last-Layer Influence Approximation: By calculating the Hessian matrix (second-order derivatives) only for the final layer, the method provides a high-quality importance signal at a fraction of the memory and time cost.
  3. Masked-Adam (M-Adam): A modified optimizer that ensures momentum statistics are only updated for samples currently being processed in a mini-batch.

Key Insights: Visualizing Data Importance

Synthetic experiments on "Circles" and "Moons" datasets provide a clear visualization of the algorithm's behavior. As training progresses, the model automatically identifies and down-weights unlabeled points near the decision boundary—where pseudo-labels are most likely to be incorrect.

Evolution of decision boundaries and per-example weights on synthetic datasets. Darker points indicate higher weights, showing how the model learns to down-weight ambiguous samples near the boundary.

This behavior demonstrates that influence functions can effectively act as "hypergradients" for per-example weights, bridging the gap between robust statistics and hyperparameter optimization. Furthermore, the method is remarkably efficient: the last-layer Hessian approximation achieved a 4.6x speedup over previous bi-level optimization methods while reducing GPU memory usage from 15.8GB to 9GB.

Experimental Results

The proposed method was evaluated across image classification (CIFAR-10, SVHN) and text classification (IMDb) benchmarks. In all cases, the per-example weighting scheme outperformed standard SSL baselines.

Comparison of test error rates across different amounts of labeled data. The proposed per-example weighting consistently outperforms both single-weight baselines and the state-of-the-art UDA method.

On CIFAR-10 with 4000 labels, the method achieved a 4.75% error rate, compared to UDA's 5.29%. On SVHN with 250 labels, it reached 2.45% error (vs. 2.76% for UDA). The gains extended to natural language processing, where the error rate on the IMDb sentiment analysis dataset was reduced to 8.51%, outperforming the 8.98% baseline. Beyond accuracy, the optimized implementation for per-example gradients proved to be 19.3x faster than a naive serial implementation.

Conclusion and Future Work

By moving beyond uniform weighting, this research demonstrates that treating unlabeled data points as unique entities significantly improves SSL performance and robustness. While the method relies on a small validation set to guide the weighting process, results show that even 64 samples are sufficient to provide a meaningful signal.

Future research may explore the application of this technique to more complex tasks like object detection or self-supervised pre-training, where the variance in data quality is even more pronounced. Despite the assumption that importance is primarily reflected in the last layer, the empirical success of this approximation suggests a powerful new tool for efficient, large-scale semi-supervised learning.

Summary updated

Abstract

Existing semi-supervised learning (SSL) algorithms use a single weight to balance the loss of labeled and unlabeled examples, i.e., all unlabeled examples are equally weighted. But not all unlabeled data are equal. In this paper we study how to use a different weight for every unlabeled example. Manual tuning of all those weights -- as done in prior work -- is no longer possible. Instead, we adjust those weights via an algorithm based on the influence function, a measure of a model's dependency on one training example. To make the approach efficient, we propose a fast and effective approximation of the influence function. We demonstrate that this technique outperforms state-of-the-art methods on semi-supervised image and language classification tasks.
BibTeX
@inproceedings{ren2020not,
  author = {Zhongzheng Ren and Raymond A. Yeh and Alexander G. Schwing},
  title = {{Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=8_BRqSXcgNr},
}
Figure