Abstract
Prompt-based learning is susceptible to intrinsic bias present in pre-trained language models (LMs), leading to sub-optimal performance in zero/few-shot settings. We propose IntrCal, a null-input prompting method to calibrate this bias. By leveraging auto-selected null-meaning inputs from GPT-4, we probe the LM's intrinsic bias and update only its bias parameters (~0.1% of total parameters) to achieve a uniform probability distribution. Experimental results show that IntrCal improves zero-shot in-context learning by an average of 9% and prompt-based fine-tuning by 2%, all while preserving the model's core language modeling capabilities.
Introduction
When Large Language Models (LLMs) are tasked with classification in zero-shot or few-shot scenarios, they often exhibit "intrinsic bias." For instance, a model might favor a "Positive" label simply because the word is more frequent in its training data, regardless of the input's actual sentiment. This bias acts as a skewed baseline, hindering the model's ability to generalize from prompts.
Previous solutions often relied on post-hoc output re-weighting, which doesn't address the model's internal state and is less effective for fine-tuning. Other retraining methods are computationally heavy and risk degrading the model's general knowledge. This paper introduces IntrCal (Intrinsic Calibration), a parameter-efficient approach to "zero" the model's internal scale before it performs a task.
The IntrCal Method
The IntrCal pipeline consists of five key stages designed to identify and mitigate internal bias without requiring human-labeled data.
- Null-Input Generation: GPT-4 generates a diverse set of "meaningless" inputs (e.g., "N/A", "An empty sentence").
- Prompt Construction & Filtering: These strings are inserted into task templates. A secondary model (like BERT) filters these prompts using Next Sentence Prediction (NSP) scores to ensure they fit naturally.
- Bias Probing: The target LM processes these null prompts. An unbiased model should distribute probability equally across all classes; any deviation indicates intrinsic bias.
- Parameter-Efficient Update: The system updates only the bias parameters ($B_{LM}$)—roughly 0.1% of the model. Using a Kullback–Leibler (KL) divergence loss, the model is nudged toward a uniform distribution for null inputs.
- One-Batch Calibration: For zero-shot tasks, researchers found that updating on a single batch (32 samples) provides the optimal balance, fixing bias without over-calibrating.
Experimental Evaluation
The researchers evaluated IntrCal across eight classification datasets covering topics (AGNews, DBPedia), sentiment (SST-5, SemEval 2014), and subjectivity (Subj, TREC). To ensure the calibration didn't "break" the model, they validated language modeling performance on WikiText and LAMBADA.
Key Results
- In-Context Learning (ICL): On RoBERTa-large, IntrCal achieved an average absolute accuracy gain of 9% in zero-shot settings.
- Prompt Fine-Tuning: The calibrated models maintained a 2% advantage even after being fine-tuned on labeled examples, suggesting that internal calibration is more robust than simple output adjustment.
- Efficiency and Generality: The method was effective across different architectures, including GPT-2 XL and Llama-2 7B. Crucially, the model's perplexity on standard language tasks remained stable, preserving its linguistic fluency.
Insights and Future Work
IntrCal demonstrates that a tiny fraction of a model's weights—the bias terms—holds significant influence over its predictive "prejudice." By establishing an equitable starting point, the model can more accurately interpret task-specific information.
Future research will look into extending this method to regression tasks and developing more automated ways to determine optimal stopping points for calibration. Additionally, as modern decoder-only models like Llama-2 often lack explicit bias parameters in some layers, finding even more efficient ways to apply IntrCal to these architectures remains a priority.
Summary updated