Abstract
Recent Active Learning (AL) approaches in Natural Language Processing (NLP) proposed using off-the-shelf pretrained language models (LMs). In this paper, we argue that these LMs are not adapted effectively to the downstream task during AL and we explore ways to address this issue. We suggest to first adapt the pretrained LM to the target task by continuing training with all the available unlabeled data and then use it for AL. We also propose a simple yet effective fine-tuning method to ensure that the adapted LM is properly trained in both low and high resource scenarios during AL. Our experiments demonstrate that our approach provides substantial data efficiency improvements compared to the standard fine-tuning approach, suggesting that a poor training strategy can be catastrophic for AL.
Introduction
Active Learning (AL) is a powerful paradigm for reducing labeling costs by iteratively selecting the most informative data points for training. While traditional AL research focused on training models from scratch, modern NLP relies on large Pretrained Language Models (LMs) like BERT. A recent study by Margatina et al., "On the Importance of Effectively Adapting Pretrained Language Models for Active Learning," argues that standard adaptation techniques are often insufficient for the low-data regimes typical of early AL iterations. They propose that the training strategy used to adapt these models is just as critical as the acquisition function used to select data.
The Challenge: Adapting LMs in Low-Data Regimes
Standard fine-tuning (SFT) of LMs can be highly unstable when labeled data is scarce. Models are typically fine-tuned for a fixed, small number of epochs, which often leads to high variance and suboptimal performance in "few-shot" scenarios. Additionally, standard AL workflows frequently overlook the potential of the large pool of unlabeled data during the training phase, using it only for data selection. This represents a missed opportunity for domain-specific adaptation.
Proposed Methodology: Effective Adaptation
To address these issues, the authors introduce a two-step adaptation framework:
- Task-Adaptive Pretraining (TAPT): Before starting the AL loop, the LM undergoes continued pretraining on the entire pool of unlabeled task data using Masked Language Modeling (MLM). This process aligns the model with the specific vocabulary and stylistic nuances of the target domain.
- Robust Fine-tuning (FT+): The authors propose a more flexible fine-tuning strategy designed to work across both low- and high-resource settings. This includes:
- Early Stopping: Training for up to 20 epochs, with termination based on validation loss to prevent overfitting.
- Adaptive Warmup: Dynamically adjusting the learning rate warmup schedule based on the current size of the labeled dataset.
Experimental Results
The TAPT-FT+ approach was evaluated on five diverse NLU benchmarks: TREC-6, DBPEDIA, IMDB, SST-2, and AGNEWS.
Data Efficiency and Performance
The results show that TAPT-FT+ consistently outperforms standard fine-tuning across all datasets. Notably, models using this approach reached full-dataset performance levels with only 15% of the total data budget. In the initial iterations (e.g., at 2% of data), TAPT-FT+ achieved accuracy gains of 12 to 45 points over SFT on datasets like TREC-6 and DBPEDIA.
Training Strategy vs. Acquisition Function
A key finding is that a robust training strategy (TAPT + FT+) often matters more than the choice of acquisition function (e.g., Entropy, BADGE, or ALPS). Even simple random sampling, when paired with TAPT-FT+, frequently outperformed more sophisticated acquisition methods that relied on standard fine-tuning.
Calibration and Robustness
Beyond raw accuracy, the proposed adaptation strategies significantly improve model calibration. An ablation study demonstrates that TAPT-FT+ leads to lower Expected Calibration Error (ECE) and Negative Log-Likelihood (NLL). This indicates that the model's confidence scores are more reliable, which is essential for uncertainty-based data selection in AL.
Conclusion
The work of Margatina et al. underscores that effective model adaptation is a prerequisite for successful Active Learning with pretrained LMs. By leveraging unlabeled data through TAPT and employing robust fine-tuning techniques, substantial data efficiency gains can be achieved, often making the specific choice of acquisition function a secondary concern. Future research may explore how these strategies scale with even larger models that possess stronger inherent few-shot capabilities.
Summary updated