Abstract
Semi-supervised active learning (SSAL) aims to minimize annotation costs by combining the strengths of semi-supervised learning (SSL) and active learning (AL). However, existing methods often treat these components as independent processes. We propose Least Pseudo-Labeling Status (LPLS), a selection strategy that tightly couples SSL and AL. By prioritizing human labeling for classes that the model struggles to pseudo-label, LPLS targets "blind spots" in the SSL process. Experimental results on text classification benchmarks show that LPLS consistently outperforms state-of-the-art baselines in both accuracy and efficiency.
Introduction
Training deep learning models for text classification typically requires large-scale labeled datasets, which are expensive to acquire. Semi-supervised learning (SSL) leverages unlabeled data by generating high-confidence "pseudo-labels," while active learning (AL) involves a human oracle to label the most informative data points.
While SSAL frameworks combine these approaches, they often lack synergy. SSL focuses on high-confidence data to expand the training set, whereas AL focuses on low-confidence data to resolve uncertainty. A critical issue arises when an SSL model performs poorly on a specific class: it fails to generate pseudo-labels for that class, creating a "blind spot" that hinders overall performance. LPLS addresses this by using the AL phase to specifically target these under-represented classes, helping the SSL phase recover and improve.
Methodology: The LPLS Framework
The LPLS strategy is built upon a multi-model version of the JointMatch SSL framework. The architecture consists of two primary components: Multi-Net uncertainty estimation and the LPLS selection strategy.
Multi-Net Uncertainty
To robustly estimate uncertainty, LPLS employs multiple differently initialized models (Multi-Nets). Uncertainty is calculated as the product of two metrics:
- Intra-model Uncertainty (Entropy): Measures the confusion of an individual model regarding a specific data point.
- Inter-model Uncertainty (Divergence): Measures the disagreement between different models using KL-Divergence.
Least Pseudo-Labeling Status (LPLS) Selection
The core innovation of LPLS is its class-aware selection logic:
- Monitoring: The system tracks the number of pseudo-labels assigned to each class during the SSL phase.
- Identification: It identifies the "needy" class—the one with the fewest pseudo-labels.
- Targeting: From the unlabeled pool, the system filters for data points that the models predict belong to this needy class.
- Querying: Among these filtered points, the one with the highest uncertainty score is selected for human labeling.
Experimental Results
The researchers evaluated LPLS on three standard text classification benchmarks: AG News (topic), Yahoo! Answer (topic), and IMDB (sentiment). LPLS was compared against AL-QBC (pure AL), JointMatch (SSL), and AcTune (SSAL).
LPLS consistently outperformed all baselines. On the AG News dataset, LPLS achieved 88.5% accuracy, surpassing JointMatch (88.1%) and AcTune (87.7%). The strategy showed particular strength in Macro-F1 scores, indicating superior handling of class distributions. Furthermore, LPLS demonstrated higher efficiency, reaching peak performance levels with fewer human-labeled samples than competing methods.
Discussion and Insights
The success of LPLS highlights the importance of synergy between SSL and AL. By labeling samples from "under-pseudo-labeled" classes, human intervention directly assists the model in breaking out of performance plateaus. Additionally, the Multi-Net approach provides a more reliable uncertainty estimation than single-model methods, as it is less likely for multiple models to be "confidently wrong" in the same way. Interestingly, the authors found that a three-model committee is often the optimal balance between diversity and computational noise.
Conclusion
LPLS offers a novel approach to semi-supervised active learning by prioritizing classes that are difficult for the model to pseudo-label. While highly effective for multi-class tasks, its impact is slightly reduced in binary classification (e.g., IMDB), where improving one class inherently aids the other. Future work will explore the application of LPLS to other domains, such as computer vision and sequence labeling, and address the computational overhead associated with running multiple models.
Summary updated