Introduction
Semi-supervised text classification (SSTC) is a critical area of research aimed at reducing the dependency on large-scale labeled datasets. By leveraging a small set of labeled examples alongside a vast pool of unlabeled data, SSTC seeks to achieve high performance with minimal human intervention. A prominent technique in this domain is pseudo-labeling, where a model generates labels for unlabeled data to expand its own training set. However, traditional pseudo-labeling methods, such as FixMatch, suffer from two primary limitations: class bias and error accumulation (confirmation bias).
Class bias occurs when fixed confidence thresholds favor "easy" classes, leading to an imbalanced training set. Error accumulation arises when a model incorporates its own incorrect high-confidence predictions into its training loop, reinforcing mistakes. To address these challenges, we present JointMatch, a holistic framework that integrates adaptive class-wise thresholds with a collaborative cross-labeling strategy between two peer networks.
Methodology
JointMatch introduces a unified architecture designed to enhance the diversity and quality of pseudo-labels through three core mechanisms:
Adaptive Local Thresholding
Unlike standard methods that employ a global fixed threshold (e.g., 0.95), JointMatch utilizes adaptive local thresholding. This approach estimates the "learning status" of each class in real-time. By dynamically lowering thresholds for difficult or underrepresented classes, the framework ensures a more diverse distribution of pseudo-labels across the entire label space from the early stages of training.
Collaborative Cross-Labeling
To mitigate the feedback loop of confirmation bias, JointMatch employs two peer networks ($f$ and $g$) with different initializations. Instead of a single model supervising itself, the networks engage in cross-labeling: Model $f$ provides pseudo-labels for Model $g$, and vice versa. This collaborative approach leverages the fact that different initializations lead to different noise patterns, allowing one network to act as a corrective filter for the other.
Weighted Disagreement and Agreement Updates
To prevent the two networks from converging too quickly and losing the benefits of collaboration, JointMatch introduces a weighting strategy. This strategy prioritizes "disagreement data"—instances where the two models predict different labels—to maintain model divergence and promote continued learning. Simultaneously, "agreement data" is retained to provide stable, high-quality supervision for the training process.
Experimental Evaluation
The authors evaluated JointMatch across a diverse range of benchmarks, including AG News (4 classes), Yahoo! Answers (10 classes), and IMDB (2 classes). To test the robustness of the framework in more complex scenarios, additional evaluations were conducted on GoEmotions (27 classes), Empathetic Dialogues (32 classes), and the Hurricane disaster-related tweet dataset (8 classes).
Performance and Data Scarcity
JointMatch demonstrates significant improvements over existing state-of-the-art (SOTA) methods, such as SAT, with an average performance increase of 5.13%. The framework is particularly effective in extreme low-data regimes. For instance, on the AG News dataset, JointMatch achieved 86% accuracy using only 5 labeled examples per class, whereas traditional methods like FixMatch struggle significantly due to the aforementioned class bias.
Ablation Studies
Ablation analysis highlights the importance of the collaborative component. Removing the cross-labeling mechanism resulted in the most substantial drop in accuracy, confirming that the interaction between peer networks is the primary driver in overcoming error accumulation.
Discussion and Limitations
While JointMatch offers a robust solution for SSTC, it introduces certain trade-offs. The primary limitation is computational overhead; training two models simultaneously roughly doubles the required GPU memory and training time (e.g., 2.5 hours vs 1.25 hours on an A5000 GPU for AG News). Additionally, the current implementation uses a fixed weight ($\delta$) for disagreement data, which could potentially be improved through an adaptive weighting scheme as the models converge.
Future work may explore the scalability of this collaborative pseudo-labeling approach to multi-billion parameter Large Language Models (LLMs) and the development of more sophisticated adaptive weighting mechanisms to further optimize the training trajectory.
Summary updated