Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Models

Kang He, Yinghan Long, Kaushik Roy

Abstract

Prompt-based learning is susceptible to intrinsic bias present in pre-trained language models (LMs), leading to sub-optimal performance in zero/few-shot settings. We propose IntrCal, a null-input prompting method to calibrate this bias. By leveraging auto-selected null-meaning inputs from GPT-4, we probe the LM's intrinsic bias and update only its bias parameters (~0.1% of total parameters) to achieve a uniform probability distribution. Experimental results show that IntrCal improves zero-shot in-context learning by an average of 9% and prompt-based fine-tuning by 2%, all while preserving the model's core language modeling capabilities.

Introduction

When Large Language Models (LLMs) are tasked with classification in zero-shot or few-shot scenarios, they often exhibit "intrinsic bias." For instance, a model might favor a "Positive" label simply because the word is more frequent in its training data, regardless of the input's actual sentiment. This bias acts as a skewed baseline, hindering the model's ability to generalize from prompts.

Previous solutions often relied on post-hoc output re-weighting, which doesn't address the model's internal state and is less effective for fine-tuning. Other retraining methods are computationally heavy and risk degrading the model's general knowledge. This paper introduces IntrCal (Intrinsic Calibration), a parameter-efficient approach to "zero" the model's internal scale before it performs a task.

The IntrCal Method

The IntrCal pipeline consists of five key stages designed to identify and mitigate internal bias without requiring human-labeled data.

Overview of the IntrCal pipeline, showing null-input generation, bias probing, and parameter-efficient calibration of the language model.
  1. Null-Input Generation: GPT-4 generates a diverse set of "meaningless" inputs (e.g., "N/A", "An empty sentence").
  2. Prompt Construction & Filtering: These strings are inserted into task templates. A secondary model (like BERT) filters these prompts using Next Sentence Prediction (NSP) scores to ensure they fit naturally.
  3. Bias Probing: The target LM processes these null prompts. An unbiased model should distribute probability equally across all classes; any deviation indicates intrinsic bias.
  4. Parameter-Efficient Update: The system updates only the bias parameters ($B_{LM}$)—roughly 0.1% of the model. Using a Kullback–Leibler (KL) divergence loss, the model is nudged toward a uniform distribution for null inputs.
  5. One-Batch Calibration: For zero-shot tasks, researchers found that updating on a single batch (32 samples) provides the optimal balance, fixing bias without over-calibrating.

Experimental Evaluation

The researchers evaluated IntrCal across eight classification datasets covering topics (AGNews, DBPedia), sentiment (SST-5, SemEval 2014), and subjectivity (Subj, TREC). To ensure the calibration didn't "break" the model, they validated language modeling performance on WikiText and LAMBADA.

Key Results

  • In-Context Learning (ICL): On RoBERTa-large, IntrCal achieved an average absolute accuracy gain of 9% in zero-shot settings.
  • Prompt Fine-Tuning: The calibrated models maintained a 2% advantage even after being fine-tuned on labeled examples, suggesting that internal calibration is more robust than simple output adjustment.
  • Efficiency and Generality: The method was effective across different architectures, including GPT-2 XL and Llama-2 7B. Crucially, the model's perplexity on standard language tasks remained stable, preserving its linguistic fluency.
Zero-shot accuracy across different datasets as a function of calibration batches, illustrating the 'one-batch' peak where performance is maximized.

Insights and Future Work

IntrCal demonstrates that a tiny fraction of a model's weights—the bias terms—holds significant influence over its predictive "prejudice." By establishing an equitable starting point, the model can more accurately interpret task-specific information.

Future research will look into extending this method to regression tasks and developing more automated ways to determine optimal stopping points for calibration. Additionally, as modern decoder-only models like Llama-2 often lack explicit bias parameters in some layers, finding even more efficient ways to apply IntrCal to these architectures remains a priority.

Summary updated

Abstract

Prompt-based learning is susceptible to intrinsic bias present in pre-trained language models (LMs), leading to sub-optimal performance in prompt-based zero/few-shot settings. In this work, we propose a null-input prompting method to calibrate intrinsic bias encoded in pre-trained LMs. Different from prior efforts that address intrinsic bias primarily for social fairness and often involve excessive computational cost, our objective is to explore enhancing LMs’ performance in downstream zero/few-shot learning while emphasizing the efficiency of intrinsic bias calibration. Specifically, we leverage a diverse set of auto-selected null-meaning inputs generated from GPT-4 to probe intrinsic bias of pre-trained LMs. Utilizing the bias-reflected probability distribution, we formulate a distribution disparity loss for bias calibration, where we exclusively update bias parameters (0.1% of total parameters) of LMs towards equal probability distribution. Experimental results show that the calibration promotes an equitable starting point for LMs while preserving language modeling abilities. Across a wide range of datasets, including sentiment analysis and topic classification, our method significantly improves zero/few-shot learning performance of LMs for both in-context learning and prompt-based fine-tuning (on average 9% and 2%, respectively).
BibTeX
@inproceedings{he2024prompt,
  author = {Kang He and Yinghan Long and Kaushik Roy},
  title = {{Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=gJ1iG3QM34},
}
Figure