Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers

Christopher Schröder, Andreas Niekler, Martin PotthastACL 2022 · DOI

Executive Summary

Active Learning (AL) for Transformers is often bottlenecked by computationally expensive query strategies. This research demonstrates that a simple, often-overlooked uncertainty-based method called Breaking Ties (BT) consistently outperforms the standard "Prediction Entropy" baseline. It offers a high-performance, "cheap" alternative for fine-tuning models like BERT with minimal labeled data, achieving near-state-of-the-art performance with a fraction of the dataset.

The Challenge: Active Learning in the Transformer Era

Active Learning is the process of strategically selecting the most informative examples for human labeling to minimize costs while maximizing model performance. While Transformers have revolutionized NLP, they present two major hurdles for AL:

  1. Computational Overhead: Many advanced AL strategies require calculating gradients for every unlabeled instance or performing complex geometric calculations. For models with hundreds of millions of parameters, this is prohibitively slow.
  2. The Default Baseline Trap: Researchers often default to "Prediction Entropy" (PE) as a simple baseline. This study re-evaluates whether PE is truly the best "simple" choice for modern architectures.

Methodology and Query Strategies

The study systematically compares five query strategies across different models, including BERT (336M parameters) and DistilRoBERTa (82M parameters):

  • Prediction Entropy (PE): Selects instances where the model is most "confused" across all possible classes.
  • Breaking Ties (BT): Selects instances where the gap between the top two most likely class predictions is the smallest.
  • Least Confidence (LC): Selects instances where the probability of the most likely class is lowest.
  • Contrastive Active Learning (CA): A complex strategy looking for instances most different from their nearest neighbors.
  • Random Sampling (RS): The control group.
Table 1: Mean Rank and Mean Result (Accuracy and AUC) for different models and strategies. BT consistently shows top-tier performance across models.

Key Insights

Breaking Ties: The New Gold Standard

For multi-class problems, BT consistently outranked PE. While mathematically identical for binary classification, BT’s focus on the ambiguity between the top two contenders proves more effective for Transformers when more than two classes are present. As shown in the performance metrics, BT often secures the best mean rank across various models.

Extreme Data Efficiency

Transformers exhibit a remarkable ability to learn from tiny samples. For instance, on the AG’s News dataset, BERT reached 90.4% accuracy using only 0.4% of the training data (525 labeled instances out of 120,000).

Figure 1: Learning curves across five benchmarks. The red line represents passive learning on the full dataset. AL strategies (especially BT and LC) rapidly approach full-data performance.

The Efficiency Paradox

Complex strategies like Contrastive Active Learning (CA) can be extremely slow. On AG's News, CA took over 1,400 seconds per query round, whereas uncertainty methods like BT took only milliseconds. The marginal performance gains of complex methods rarely justify the massive increase in compute time.

Results and Practicality

  • Mean Rank: Across all experiments, Breaking Ties (BT) achieved the best mean rank in both final accuracy and Area Under the Learning Curve (AUC) for most models.
  • Performance vs. Full Training: In most datasets, AL models using only a few hundred samples approached or exceeded the performance of models trained on the entire dataset.
  • DistilRoBERTa: This model achieved results close to the full BERT model while using less than 25% of the parameters, making it a highly viable "budget" option for practitioners.

Limitations and Future Work

The study focuses on sentence-level text classification. Future research should explore if these simple uncertainty strategies remain effective for more complex tasks like Named Entity Recognition (NER) or Abstractive Summarization. Additionally, investigating better "warm-up" strategies for the initial "cold start" phase could further enhance early learning curves. While BT performed well generally, the complex CA strategy showed advantages on imbalanced datasets like TREC-6, suggesting uncertainty alone might struggle with heavily skewed distributions.

Summary updated

BibTeX
@inproceedings{schroder2022revisiting,
  author = {Christopher Schröder and Andreas Niekler and Martin Potthast},
  title = {{Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.findings-acl.172},
}
Figure