NP-Match: When Neural Processes meet Semi-Supervised Learning

Jianfeng Wang, Thomas Lukasiewicz, Daniela Massiceti, Xiaolin Hu, Vladimir Pavlovic, Alexandros Neophytou

Abstract

Semi-supervised learning (SSL) has been widely explored in recent years as an effective way of leveraging unlabeled data to reduce reliance on labeled data. In this work, we adjust neural processes (NPs) to the semi-supervised image classification task, resulting in a new method named NP-Match. NP-Match is suited to this task for two reasons. Firstly, NP-Match implicitly compares data points when making predictions; the prediction of each unlabeled data point is affected by the labeled data points that are similar to it, which improves the quality of pseudo-labels. Secondly, NP-Match is able to estimate uncertainty that can be used as a tool for selecting unlabeled samples with reliable pseudo-labels. Compared with uncertainty-based SSL methods implemented with Monte Carlo (MC) dropout, NP-Match estimates uncertainty with much less computational overhead. We conducted extensive experiments on four public datasets, and NP-Match outperforms state-of-the-art (SOTA) results or achieves competitive results on them, showing its potential for SSL.

Introduction

In the era of deep learning, the scarcity of labeled data remains a significant bottleneck. Semi-supervised learning (SSL) addresses this by utilizing large pools of unlabeled data. Most current SOTA methods, such as FixMatch, are deterministic, relying on a single prediction and a confidence threshold to filter pseudo-labels. However, these models can be overconfident in erroneous predictions. While probabilistic models like Bayesian Neural Networks can estimate uncertainty to mitigate this, they often rely on computationally expensive techniques like Monte Carlo (MC) Dropout, which requires multiple forward passes. NP-Match bridges this gap by providing a probabilistic framework that is both accurate and computationally efficient.

The NP-Match Architecture

NP-Match replaces the standard classification head with a Neural Process (NP) model. NPs combine the flexibility of neural networks with the formal uncertainty handling of Gaussian Processes.

The NP-Match framework overview showing Training and Inference modes.

As shown in the architecture above, the model consists of several key stages:

  1. Feature Extraction: A CNN backbone extracts feature vectors from both context (labeled) and target (unlabeled) points.
  2. Dual Paths: The Latent Path captures global uncertainty by encoding context points into a distribution, while the Deterministic Path captures local information.
  3. Implicit Comparison: By treating labeled data as context points, the model learns to predict the label of a target image by implicitly comparing it to known examples, leading to higher-quality pseudo-labels.
  4. JS Divergence: A major contribution is the Uncertainty-Guided Skew-Geometric JS Divergence ($JS^{G\alpha_u}$). This measure uses the model's own uncertainty to dynamically adjust the learning process, preventing noisy pseudo-labels from polluting the latent space.

Experimental Results and Efficiency

NP-Match was evaluated on CIFAR-10, CIFAR-100, STL-10, and ImageNet. The results demonstrate significant improvements, particularly in low-label regimes. For instance, on CIFAR-10 with only 40 labels, NP-Match achieved a 4.91% error rate, significantly lower than FixMatch’s 7.47%.

Uncertainty and accuracy metrics for the STL-10 dataset across different numbers of labeled samples.

The performance on the STL-10 dataset is particularly noteworthy. STL-10 is challenging due to out-of-distribution images in its unlabeled set. NP-Match reached a 14.20% error rate with 40 labels, outperforming the previous SOTA, FlexMatch, which stood at 29.15%. As illustrated in the figure above, the model maintains high accuracy across classes while providing calibrated uncertainty scores that act as a gatekeeper for pseudo-label selection.

Beyond accuracy, NP-Match offers substantial efficiency gains. Unlike MC Dropout, which requires $T$ full forward passes through the entire network, NP-Match only samples from the latent distribution at the end of the pipeline. This allows the computational cost to remain nearly flat even as the number of samples for uncertainty estimation increases.

Conclusion

NP-Match introduces a robust probabilistic framework for SSL that leverages the strengths of Neural Processes. By providing efficient uncertainty estimation and high-quality pseudo-labels through implicit data comparison, it sets a new benchmark for semi-supervised image classification, especially when labeled data is extremely scarce. Future work may involve extending this framework to more complex tasks like object detection and semantic segmentation.

Summary updated

Abstract

Semi-supervised learning (SSL) has been widely explored in recent years, and it is an effective way of leveraging unlabeled data to reduce the reliance on labeled data. In this work, we adjust neural processes (NPs) to the semi-supervised image classification task, resulting in a new method named NP-Match. NP-Match is suited to this task for two reasons. Firstly, NP-Match implicitly compares data points when making predictions, and as a result, the prediction of each unlabeled data point is affected by the labeled data points that are similar to it, which improves the quality of pseudo-labels. Secondly, NP-Match is able to estimate uncertainty that can be used as a tool for selecting unlabeled samples with reliable pseudo-labels. Compared with uncertainty-based SSL methods implemented with Monte Carlo (MC) dropout, NP-Match estimates uncertainty with much less computational overhead, which can save time at both the training and the testing phases. We conducted extensive experiments on four public datasets, and NP-Match outperforms state-of-the-art (SOTA) results or achieves competitive results on them, which shows the effectiveness of NP-Match and its potential for SSL.
BibTeX
@inproceedings{wang2022np,
  author = {Jianfeng Wang and Thomas Lukasiewicz and Daniela Massiceti and Xiaolin Hu and Vladimir Pavlovic and Alexandros Neophytou},
  title = {{NP-Match: When Neural Processes meet Semi-Supervised Learning}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=1Ip1Fxu86Zy},
}
Figure