Neural Self-Training through Spaced Repetition

Hadi AmiriNAACL 2019 · OpenReview

Abstract

Self-training is a semi-supervised learning approach for utilizing unlabeled data to create better learners. The efficacy of self-training algorithms depends on their data sampling techniques. The majority of current sampling techniques are based on predetermined policies which may not effectively explore the data space or improve model generalizability. In this work, we tackle the above challenges by introducing a new data sampling technique based on spaced repetition that dynamically samples informative and diverse unlabeled instances with respect to individual learner and instance characteristics. The proposed model is specifically effective in the context of neural models which can suffer from overfitting and high-variance gradients when trained with small amount of labeled data. Our model outperforms current semi-supervised learning approaches developed for neural networks on publicly-available datasets.

Introduction

Deep learning models are notoriously data-hungry, yet the cost of manual labeling remains a significant bottleneck. Self-training offers a popular semi-supervised solution: a model is trained on a small labeled set, predicts labels for a larger unlabeled set (pseudo-labels), and incorporates the most "confident" predictions back into its training pool.

However, standard self-training often falls into a "confidence trap." By prioritizing the easiest samples—those the model is already certain about—the algorithm reinforces existing biases rather than exploring new, informative regions of the data space. This paper introduces Leitner Self-Training (LST), which utilizes the Leitner system of spaced repetition to intelligently sample diverse and informative instances.

Methodology: The Leitner System for Data Sampling

The core innovation is the integration of the Leitner system, a flashcard-based learning strategy from cognitive psychology. In LST, unlabeled data instances are organized into a series of queues ($q_0, q_1, \dots, q_n$).

The proposed Leitner Self-Training (LST) architecture, which dynamically manages unlabeled data through a series of queues to optimize sampling.

Promotion and Demotion

All unlabeled data begins in the first queue ($q_0$). During training, if the model correctly predicts an instance's pseudo-label, the instance is "promoted" to a higher queue. If the prediction is incorrect or lacks sufficient confidence, it is "demoted" to a lower queue. This movement naturally decouples an instance's "easiness" from its "informativeness."

The Designated Queue

Instead of sampling from the highest-confidence (easiest) queue, LST identifies a "designated queue"—the specific group of instances that, when added to the training set, results in the largest performance gain on a validation set. This approach allows the model to find the "sweet spot" of data that is challenging enough to provide new information without being so noisy as to degrade performance.

Experimental Results

The authors evaluated LST on two text classification tasks: IMDb (sentiment analysis) and Churn (predicting user intent to leave a brand via tweets).

  • IMDb: LST achieved a 78.27 macro-F1, significantly outperforming the base classifier (73.02) and the best previous baseline, Knowledge Transfer ST (77.11).
  • Churn: LST reached a 69.90 macro-F1, a notable improvement over the base classifier (65.77) and Adversarial ST (67.70).
Macro-F1 performance across different Leitner queues. The peak in the middle queues (e.g., $q_2$) illustrates that the most informative samples are often those the model is currently "learning" rather than those it has already mastered.

Key Insights

The "Sweet Spot" is in the Middle

As shown in the performance analysis across queues, the most useful instances are typically found in the middle or upper-middle queues. These instances exhibit high "movement deviation," frequently shifting between promotion and demotion. This volatility suggests these samples lie near the model's decision boundary and are highly informative for the model's growth.

Diversity vs. Confidence

Standard self-training tends to sample redundant data that closely matches the existing training set. LST's queue-based approach ensures that the sampled data is diverse. By avoiding the highest-confidence samples, which often offer little new information, LST achieves better generalizability on unseen data.

Conclusion

Leitner Self-Training provides a robust framework for semi-supervised learning by adapting to the specific learning trajectory of the model. By leveraging spaced repetition to identify the most informative data points, LST overcomes the limitations of traditional self-training and sets a new benchmark for neural semi-supervised learning on text classification tasks.

Summary updated

Abstract

Self-training is a semi-supervised learning approach for utilizing unlabeled data to create better learners. The efficacy of self-training algorithms depends on their data sampling techniques. The majority of current sampling techniques are based on predetermined policies which may not effectively explore the data space or improve model generalizability. In this work, we tackle the above challenges by introducing a new data sampling technique based on spaced repetition that dynamically samples informative and diverse unlabeled instances with respect to individual learner and instance characteristics. The proposed model is specifically effective in the context of neural models which can suffer from overfitting and high-variance gradients when trained with small amount of labeled data. Our model outperforms current semi-supervised learning approaches developed for neural networks on publicly-available datasets.
BibTeX
@inproceedings{amiri2019neural,
  author = {Hadi Amiri},
  title = {{Neural Self-Training through Spaced Repetition}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
  url = {https://openreview.net/forum?id=BJEYNXbuZr},
}
Figure