Abstract
Active learning, which effectively collects informative unlabeled data for annotation, reduces the demand for labeled data. In this work, we propose to retrieve unlabeled samples with a local sensitivity and hardness-aware acquisition function. The proposed method generates data copies through local perturbations and selects data points whose predictive likelihoods diverge the most from their copies. We further empower our acquisition function by injecting the select-worst case perturbation. Our method achieves consistent gains over the commonly used active learning strategies in various classification tasks. Furthermore, we observe consistent improvements over the baselines on the study of prompt selection in prompt-based few-shot learning. These experiments demonstrate that our acquisition guided by local sensitivity and hardness can be effective and beneficial for many NLP tasks.
Introduction
In modern machine learning, the performance of models is often gated by the availability of high-quality labeled data. While unlabeled data is abundant, the human effort required for annotation is expensive and time-consuming. Active Learning (AL) seeks to optimize this process by identifying the most informative examples from an unlabeled pool for human labeling.
Traditional AL strategies typically fall into two categories: Uncertainty Sampling, which targets examples where the model lacks confidence, and Diversity Sampling, which ensures the selected data represents the entire distribution. However, these methods have pitfalls; uncertainty sampling often gravitates toward noisy outliers, while diversity sampling may select "easy" examples that provide little new information. ALLSH (Active Learning guided by Local Sensitivity and Hardness) introduces a third perspective: identifying samples located near the model's decision boundaries by measuring local sensitivity.
Methodology
The core of the ALLSH framework is an acquisition function that identifies samples where the model’s predictions are inconsistent within a small neighborhood. The process follows four primary steps:
- Local Perturbation (Data Paraphrasing): For each unlabeled sentence, the system generates multiple "copies" using augmentation techniques such as TF-IDF word replacement and back-translation (e.g., English to German and back to English).
- Divergence Measurement: The original sentence and its augmented copies are passed through the model. The system calculates the Kullback–Leibler (KL) Divergence between the predicted probability distributions of the original and the copies.
- Worst-Case Augmentation (WCA): Rather than averaging the divergence, ALLSH identifies the "worst-case" copy—the one that causes the model's prediction to deviate most significantly from the original. This serves as a computationally efficient proxy for adversarial searching in discrete text space.
- Ranking and Selection: Examples exhibiting the highest divergence (the most sensitivity) are prioritized for labeling.
To further enhance performance, the authors incorporate Curriculum Learning (CL), using a semi-supervised loss function that encourages the model to maintain consistency across these local regions during the training phase.
Experimental Setup
The researchers evaluated ALLSH across several Natural Language Processing (NLP) benchmarks to test its versatility:
- Text Classification: SST-2 and IMDB (Sentiment Analysis), AG News (Topic Classification), QNLI (Natural Language Inference), and QQP (Paraphrase Detection).
- Out-of-Domain (OD) Generalization: Testing models trained on one dataset (e.g., SST-2) against different but related datasets (e.g., IMDB) to evaluate robustness.
- Prompt-based Few-Shot Learning: Utilizing datasets like TREC and RTE to determine if ALLSH can effectively select optimal "in-context" examples for large language models like GPT-2.
Key Insights and Results
The "Goldilocks" Zone of Informativeness
ALLSH identifies a "sweet spot" for data selection. By focusing on local sensitivity, the method avoids the sampling bias of standard uncertainty methods. It does not merely pick the most difficult examples (which may be outliers or gibberish) but rather focuses on examples where the model is "almost" correct but easily swayed by minor linguistic variations. This suggests that sensitivity is a more reliable signal for informativeness than simple low confidence.
Performance Gains
- Data Efficiency: On the SST-2 sentiment task, ALLSH achieved 85.97% accuracy using only 1% of the labeled data. This outperformed both Random sampling (84.11%) and standard Entropy-based uncertainty sampling (84.53%).
- Robustness: In out-of-domain testing (SST-2 to IMDB), ALLSH provided a 4.2% improvement over the random baseline. This indicates that the data selected by ALLSH helps the model learn more fundamental, generalizable linguistic features rather than dataset-specific shortcuts.
- Prompt Selection: In few-shot scenarios with GPT-2, ALLSH consistently outperformed "Calibrated" or "Uncertainty" methods, yielding gains of 2-5% in 4-shot and 8-shot settings.
Computational Efficiency
While more intensive than simple entropy calculations, ALLSH is significantly more efficient than complex gradient-based methods like BADGE. By using discrete augmentations instead of expensive gradient computations and clustering, ALLSH provides a scalable alternative for large-scale datasets.
Limitations and Future Work
The effectiveness of ALLSH is inherently tied to the quality of the data augmentation. If back-translation or word replacement alters the semantic meaning of a sentence (e.g., changing a positive sentiment to negative), the sensitivity signal becomes misleading noise. Additionally, because text is discrete, the "worst-case" search is limited by the diversity of the available augmentation tools.
Future research may explore learnable augmentation strategies that can more precisely probe a model's decision boundaries, potentially further reducing the amount of human intervention required to train high-performing NLP models.
Summary updated