Pseudo Label Selection is a Decision Problem

Julian RodemannarXiv 2023 · OpenReview

TL;DR

This paper reframes pseudo-label selection (PLS) in semi-supervised learning as a decision problem, leading to a novel Bayesian framework (BPLS). BPLS offers a theoretically grounded criterion that mitigates "confirmation bias" (propagating initial model errors) and improves robustness, especially in scenarios prone to overfitting.

What Problem Does This Paper Address?

Semi-supervised learning (SSL) is crucial when labeled data is scarce. Pseudo-labeling (or self-training) is a popular SSL technique where a model trained on labeled data is used to predict labels for unlabeled data, and then these "pseudo-labels" are added to the training set. However, the selection of which pseudo-labeled instances to use is critical and often lacks theoretical justification. A major pitfall is "confirmation bias": if the initial model overfits, it can confidently predict incorrect labels for some unlabeled data, and adding these to the training set can propagate and amplify these errors, hindering generalization. This paper addresses the need for a principled, robust method for selecting pseudo-labels that avoids this confirmation bias.

Methods

The core contribution is framing pseudo-label selection (PLS) as a decision problem. Instead of ad-hoc criteria, the authors propose a Bayesian Pseudo-Label Selection (BPLS) framework.

  • Decision-Theoretic Embedding: PLS is re-conceptualized as choosing actions (selecting unlabeled data points) within a decision-theoretic framework.
  • Bayes-Optimal Criterion: The key innovation is a novel selection criterion derived from the Bayes-optimality of the "pseudo posterior predictive." This criterion is an analytical approximation of the posterior predictive of pseudo-samples and labeled data.
  • Approximation Technique: To avoid computationally expensive sampling, they use approximations based on Laplace's method and Gaussian integrals. The resulting criterion is l(θ) - log|I(θ)|, where l(θ) is the likelihood and I(θ) is the Fisher information matrix at the fitted parameter vector θ. This approximation is surprisingly robust and doesn't require an i.i.d. assumption.
  • Robustness Extension: A second contribution extends BPLS to be more robust against modeling assumptions. This is achieved by defining a multi-objective utility function that can incorporate different sources of uncertainty, such as model selection, error accumulation, and covariate shift.

Datasets

The paper mentions using simulated and real-world data for evaluation. Specific datasets are not detailed, but the evaluation covers generalized linear models, non-parametric generalized additive models, and Bayesian neural networks.

Key Insights and Ideas

  • PLS as a Decision Problem: This reframing provides a strong theoretical foundation for pseudo-label selection, moving beyond heuristic approaches.
  • Confirmation Bias Mitigation: The proposed Bayesian approach directly tackles confirmation bias by using a criterion that is less susceptible to initial model overfitting.
  • Analytical Bayes-Optimality: Deriving a computationally tractable, Bayes-optimal criterion for PLS is a significant theoretical advancement.
  • Robustness through Multi-Objective Utilities: The ability to define utility functions that account for various uncertainties (model choice, data shifts) makes the PLS process more adaptable and reliable.

Key Results

  • Outperformance: BPLS outperforms traditional PLS methods, especially on data prone to overfitting and confirmation bias.
  • Computational Efficiency: The approximate Bayes-optimal criterion is computationally cheap to evaluate.
  • Flexibility: BPLS can incorporate prior knowledge and does not require hyperparameter tuning.
  • Robustness Gains: Extensions of BPLS demonstrate substantial performance gains, particularly in robustness with regard to model choice.

Limitations and Future Work

  • Approximation Accuracy: While the approximation is effective, its accuracy might vary depending on the specific model and data distribution.
  • Multi-Objective Utility Design: Designing effective multi-objective utility functions for complex real-world scenarios could be challenging.
  • Future Work: The authors suggest exploring further extensions of the decision-theoretic framework for PLS, including more sophisticated ways to handle uncertainty and model misspecification. The paper also hints at connections to optimistic/pessimistic superset learning.

Summary updated

Abstract

Pseudo-Labeling is a simple and effective approach to semi-supervised learning. It requires criteria that guide the selection of pseudo-labeled data. The latter have been shown to crucially affect pseudo-labeling's generalization performance. Several such criteria exist and were proven to work reasonably well in practice. However, their performance often depends on the initial model fit on labeled data. Early overfitting can be propagated to the final model by choosing instances with overconfident but wrong predictions, often called confirmation bias. In two recent works, we demonstrate that pseudo-label selection (PLS) can be naturally embedded into decision theory. This paves the way for BPLS, a Bayesian framework for PLS that mitigates the issue of confirmation bias. At its heart is a novel selection criterion: an analytical approximation of the posterior predictive of pseudo-samples and labeled data. We derive this selection criterion by proving Bayes-optimality of this "pseudo posterior predictive". We empirically assess BPLS for generalized linear, non-parametric generalized additive models and Bayesian neural networks on simulated and real-world data. When faced with data prone to overfitting and thus a high chance of confirmation bias, BPLS outperforms traditional PLS methods. The decision-theoretic embedding further allows us to render PLS more robust towards the involved modeling assumptions. To achieve this goal, we introduce a multi-objective utility function. We demonstrate that the latter can be constructed to account for different sources of uncertainty and explore three examples: model selection, accumulation of errors and covariate shift.
BibTeX
@misc{rodemann2023pseudo,
  author = {Julian Rodemann},
  title = {{Pseudo Label Selection is a Decision Problem}},
  year = {2023},
  eprint = {2309.13926},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.13926},
}

Loading related content...

Figure