Pathologies of Pre-trained Language Models in Few-shot Fine-tuning

Hanjie Chen, Guoqing Zheng, Ahmed Hassan Awadallah, Yangfeng Ji

Abstract

While few-shot fine-tuning enables pre-trained language models (PLMs) to achieve high accuracy with minimal data, the underlying mechanisms of this success remain opaque. This study investigates whether these gains stem from genuine semantic understanding or "pathological" shortcuts. Using post-hoc explanations and statistical metrics, we demonstrate that PLMs often rely on non-task-related features and shallow patterns to mitigate inherent pre-training biases, raising concerns about their robustness and generalization in real-world applications.

Introduction

Pre-trained language models (PLMs) like BERT and RoBERTa are remarkably effective at few-shot learning—adapting to new tasks using only a handful of examples. However, the research community has largely treated this success as a "black box." While accuracy metrics improve, it is unclear whether the models are learning the underlying logic of a task (such as sentiment or entailment) or simply identifying statistical "cheats" within the small training set. This paper investigates whether the impressive performance of few-shot fine-tuning is built on a foundation of genuine understanding or pathological behavior.

Methodology

To analyze the internal decision-making of these models, the authors employ post-hoc explanations via Sampling Shapley values. This technique identifies which specific tokens contribute most to a model's prediction. To analyze these explanations at scale, the study introduces several metrics:

  • Local Mutual Information (LMI): Used to identify features statistically associated with specific labels (e.g., which words the model associates with "Positive" sentiment).
  • Prediction Bias (PB): A metric designed to quantify how much a model favors one label over others.
  • KL Divergence: Used to track the distribution of features between the pre-trained model, the fine-tuned model, and the training data, visualizing the shift in the model's "reasoning" during the fine-tuning process.

Experimental Setup

The study evaluates BERT and RoBERTa across three classic NLP tasks, utilizing both in-domain (ID) and out-of-domain (OOD) datasets to test generalization:

  • Sentiment Classification: IMDB (ID) and Yelp (OOD).
  • Natural Language Inference (NLI): SNLI (ID) and MNLI (OOD).
  • Paraphrase Identification: QQP (ID) and TPPDB (OOD).

Pathological Behaviors in Few-shot Learning

The investigation revealed several critical pathologies that emerge during the fine-tuning process:

1. The Majority Label Trap

Before fine-tuning, PLMs exhibit extreme inherent bias. For instance, BERT defaults to predicting "Positive" for nearly all IMDB reviews regardless of content. This initial state represents a significant hurdle that the model must overcome during training.

2. Pathological "Fixes" and Shortcut Features

Fine-tuning with limited data does mitigate the initial bias, but often for the wrong reasons. In the early stages of training, models frequently latch onto "junk" features—such as stop words ("the", "a") or random sub-word fragments—to distinguish between labels rather than learning semantic cues.

3. Asynchronous Learning

Models do not learn all classes simultaneously. Instead, they focus on adjusting behavior for "minority" labels (the classes they initially ignored) first. This suggests the model is performing a corrective shift rather than developing a holistic representation of the task.

4. Shallow Pattern Matching

In sentence-pair tasks like NLI, models often ignore the actual meaning of the sentences. Instead, they rely on "lexical overlap"—predicting "entailment" simply because the two sentences share many of the same words.

Key Results and Generalization

The study found that while fine-tuning effectively reduces bias—for example, BERT's Prediction Bias (PB) on IMDB dropped from 0.97 to 0.05 after fine-tuning on just 1% of the data—the quality of the learned features is inconsistent.

At the start of training ($r=0$), top features for negative sentiment were often nonsensical (e.g., ##zog). Only as training progressed ($r=0.5$) did the model begin to identify task-relevant words like "bad" or "terrible." Furthermore, while accuracy on in-domain tasks (like SNLI) improved rapidly, the models struggled significantly on out-of-domain tasks (like MNLI). This "generalization gap" confirms that the shortcuts learned in few-shot settings often fail to hold up in more complex, real-world scenarios.

Conclusion

The high accuracy observed in few-shot fine-tuning can be an illusion created by shallow pattern matching. The authors conclude that the research community requires more rigorous evaluation designs and "sanity checks" that go beyond simple accuracy. Future work should investigate whether alternative methods, such as prompt-based fine-tuning, can avoid these pathological behaviors and lead to more robust semantic understanding.

Summary updated

Abstract

Although adapting pre-trained language models with few examples has shown promising performance on text classification, there is a lack of understanding of where the performance gain comes from. In this work, we propose to answer this question by interpreting the adaptation behavior using post-hoc explanations from model predictions. By modeling feature statistics of explanations, we discover that (1) without fine-tuning, pre-trained models (e.g. BERT and RoBERTa) show strong prediction bias across labels; (2) although few-shot fine-tuning can mitigate the prediction bias and demonstrate promising prediction performance, our analysis shows models gain performance improvement by capturing non-task-related features (e.g. stop words) or shallow data patterns (e.g. lexical overlaps). These observations alert that pursuing model performance with fewer examples may incur pathological prediction behavior, which requires further sanity check on model predictions and careful design in model evaluations in few-shot fine-tuning.
BibTeX
@inproceedings{chen2022pathologies,
  author = {Hanjie Chen and Guoqing Zheng and Ahmed Hassan Awadallah and Yangfeng Ji},
  title = {{Pathologies of Pre-trained Language Models in Few-shot Fine-tuning}},
  booktitle = {Insights @ ACL},
  year = {2022},
  doi = {10.18653/v1/2022.insights-1.20},
}
Figure