Cold-Start Data Selection for Better Few-shot Language Model Fine-tuning: A Prompt-based Uncertainty Propagation Approach

Yue Yu, Rongzhi Zhang, Ran Xu, Jieyu Zhang, Jiaming Shen, Chao Zhang

Introduction

The performance of Large Language Models (LLMs) in few-shot scenarios is highly dependent on the quality and representativeness of the selected training examples. While Active Learning (AL) aims to identify the most informative samples for labeling, traditional AL methods often encounter a "cold-start" problem: they require a significant amount of initial labeled data to effectively estimate sample importance. In many practical applications, practitioners begin with entirely unlabeled datasets and a limited annotation budget, making traditional AL impractical.

To address this, the PATRON (Prompt-based dAta selecTiOn for few-shot PLM fiNe-tuning) framework introduces a novel approach to cold-start data selection. By leveraging prompt-based uncertainty and a "partition-then-rewrite" strategy, PATRON identifies high-value samples for labeling without requiring any initial training data.

Methodology

The PATRON framework is built upon three core pillars designed to maximize the information gain from a minimal labeling budget.

1. Prompt-based Uncertainty Estimation

In the absence of initial labels, PATRON utilizes the zero-shot capabilities of Pre-trained Language Models (PLMs). By converting classification tasks into "cloze-style" prompts (e.g., "The movie was [MASK]."), the framework uses the model's prediction for the mask token to generate pseudo-labels. The entropy of these predictions serves as an initial measure of uncertainty, identifying samples where the model's pre-existing knowledge is most ambiguous.

2. Uncertainty Propagation

Zero-shot pseudo-labels can be inherently noisy. PATRON mitigates this by applying a kernel similarity approach to "smooth" uncertainty across the embedding space. This process, known as uncertainty propagation, ensures that a sample is only prioritized if both the sample itself and its neighbors in the embedding space exhibit high uncertainty. This prevents the selection of outliers or "noisy" points that do not represent broader regions of model weakness.

3. Partition-then-Rewrite (PTR)

To ensure diversity and minimize redundancy in the selected set, PATRON employs a Partition-then-Rewrite (PTR) strategy. The unlabeled data is first partitioned into clusters using K-Means on SimCSE embeddings. An iterative rewriting process then refines the selection within these clusters. This strategy maximizes uncertainty while explicitly penalizing the selection of points that are geographically close to samples already chosen in neighboring clusters, ensuring a broad coverage of the data distribution.

Experimental Evaluation

The researchers evaluated PATRON across six diverse text classification benchmarks, including sentiment analysis (IMDB, Yelp-full), topic classification (AG News, Yahoo! Answers, DBPedia), and question typing (TREC). Additionally, the framework's robustness was tested against Out-of-Distribution (OOD) datasets such as SST-2 and the IMDB Counterfactually Augmented Dataset.

Key Findings

The experimental results demonstrate the efficacy of PATRON in low-resource settings:

  • High Performance with Minimal Labels: With only 128 labeled examples, PATRON achieved over 91% of the performance of models trained on full, supervised datasets containing tens of thousands of samples.
  • Superiority Over Baselines: PATRON consistently outperformed existing cold-start active learning baselines, such as ALPS and TPC, by average margins ranging from 3.2% to 6.9%.
  • Increased Stability: Unlike random sampling, which is prone to high variance in few-shot settings, PATRON provided stable and predictable performance gains.
  • Generalization and Robustness: Models fine-tuned on PATRON-selected samples showed improved generalization to OOD data, suggesting that the framework identifies core representative samples rather than dataset-specific artifacts.

Conclusion and Future Directions

PATRON represents a significant advancement in cold-start data selection, bridging the gap between zero-shot prompting and supervised fine-tuning. By utilizing uncertainty propagation and diversity-focused partitioning, it enables highly efficient model adaptation with minimal human intervention.

Future research may explore the application of PATRON to generative architectures (e.g., GPT-style models) beyond the Masked Language Models (MLMs) used in this study. Additionally, extending the framework to more complex tasks such as sequence labeling or text generation remains a promising avenue for development.

Summary updated

BibTeX
@inproceedings{yu2023cold,
  author = {Yue Yu and Rongzhi Zhang and Ran Xu and Jieyu Zhang and Jiaming Shen and Chao Zhang},
  title = {{Cold-Start Data Selection for Better Few-shot Language Model Fine-tuning: A Prompt-based Uncertainty Propagation Approach}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.141},
}
Figure