TL;DR
Large Language Models (LLMs) often rely on their pre-existing knowledge (semantic priors) rather than the provided examples when performing tasks via In-Context Learning (ICL). This paper introduces "In-Context Calibration" to address this "Demonstration Shortcut," enabling LLMs to better learn new input-label relationships from demonstrations. The method shows significant improvements across various LLM families and tasks, especially in settings where novel label mappings are introduced.
What Problem Does This Paper Address?
In-Context Learning (ICL) is a powerful capability of Large Language Models (LLMs) where they can perform new tasks by observing a few examples (demonstrations) without any parameter updates. However, a key challenge is that LLMs tend to rely on their pre-trained semantic knowledge associated with the words in the demonstrations, rather than truly learning the specific input-label mapping presented. This phenomenon, termed the "Demonstration Shortcut," means the LLM might predict based on what the words usually mean, rather than what they are meant to mean in the context of the task. This issue is particularly problematic when LLMs need to learn entirely new mappings, such as when the label space is semantically unrelated to the pre-training data. Previous work has focused on improving ICL performance but often overlooks this reliance on semantic priors, especially in scenarios requiring genuine task learning.
Methods
The core contribution of this paper is the In-Context Calibration method. It aims to rectify the "Demonstration Shortcut" by explicitly estimating and mitigating the LLM's reliance on semantic priors from demonstrations. The method involves two key steps:
- Estimating Demonstration Semantic Priors: For each demonstration example, the method estimates its semantic distribution given the remaining demonstrations. This is done by using the LLM to predict the probability of the demonstration's label given its input, while also considering the context of other demonstrations.
- Estimating Context-Agnostic Semantic Priors: To further isolate the semantic priors, the input of each demonstration is randomly shuffled. This process, denoted by $R(x_i)$, aims to strip away grammatical context and retain only the semantic meaning of the words, allowing for an estimation of the LLM's pre-trained semantic knowledge for that specific input.
These two estimated priors are then combined using a hyperparameter $\lambda$ to form a demonstration-aware calibration term. This term is used to adjust the LLM's predictions, guiding it to prioritize the learned input-label relationships over its pre-trained semantic biases.
Datasets
The authors evaluated their method on 27 classification datasets spanning three categories:
- Sentiment Tasks: AGNews, CR, financial_phrasebank, poem_sentiment, MR, sst2, Subj, TREC.
- Natural Language Inference (NLI) Tasks: ANLI, SICK, RTE, WNLI, CB.
- Detection Tasks: social_bias_frames, tweet_eval_hate, tweet_eval_irony, tweet_eval_offensive, tweet_eval_stance_athesim, tweet_eval_stance_feminist, ethos_binary, ethos_disability, ethos_gender, ethos_national_origin, ethos_race, ethos_religion, ethos_violence, hate_speech18.
The experiments were conducted in two settings:
- Original ICL Task: Using the standard label space for each dataset.
- Task Learning Setting: Where the label space was replaced with semantically unrelated tokens (specifically, string numbers) to force the LLM to learn novel input-label mappings.
Key Insights and Ideas
- The "Demonstration Shortcut" is a real and impactful phenomenon: The paper provides compelling evidence that LLMs don't always learn from demonstrations as intended, often falling back on their pre-trained semantic knowledge. This is particularly evident in the "Task Learning" setting where novel label mappings are introduced.
- Semantic priors are a significant hurdle for task learning: Even when demonstrations have identical label orders and uniform distributions, changing the semantics of the examples can drastically alter the LLM's predictions, highlighting the strong influence of pre-trained knowledge.
- Calibration needs to be "demonstration-aware": Existing calibration methods that rely on content-free tokens or test set distributions are insufficient because they don't account for the specific semantic biases introduced by the demonstrations themselves.
- In-Context Calibration effectively disentangles semantic priors from task learning: By explicitly modeling and mitigating semantic priors, the proposed method allows LLMs to better learn the intended input-label relationships, leading to improved performance.
- The method generalizes across LLM families and sizes: The effectiveness of In-Context Calibration is demonstrated across OPT, GPT, and Llama2 families, and across various model sizes, suggesting its broad applicability.
Key Results
- Substantial improvements in Original ICL: For Llama2, In-Context Calibration achieved an average F1 score improvement of 23% compared to original inference in the Original ICL Task. For GPT, the improvement was 27%.
- Significant gains in Task Learning: In the Task Learning setting, where novel mappings were crucial, In-Context Calibration consistently outperformed baselines. For Llama2 70B, it improved F1-score by 9% in Task Learning.
- Outperforms baselines: In-Context Calibration consistently outperformed Contextual Calibration (CC) and Domain Calibration (DC) across various tasks and models, especially in challenging NLI tasks.
- Robustness across model families and sizes: The method showed consistent improvements across OPT, GPT, and Llama2 families, from smaller models (e.g., 2.7B) to larger ones (e.g., 70B).
- Effective even with instruction-tuned models: The paper shows that In-Context Calibration remains effective even after instruction tuning, which is known to strengthen the reliance on semantic priors.
Limitations and Future Work
- Focus on classification tasks: The current work primarily focuses on classification tasks. The authors acknowledge that the "Demonstration Shortcut" might manifest differently in generation tasks, and adapting the method for such tasks is a direction for future research.
- Computational constraints: Experiments with larger models (e.g., GPT-4 API) and multilingual settings were not feasible due to budget constraints. Future studies with more resources could provide a more comprehensive understanding.
- Hyperparameter $\lambda$ exploration: While the paper provides guidelines for selecting $\lambda$, exploring every possible value for every model was impractical. The authors believe their comprehensive analysis in the appendix offers practical guidance.
The authors anticipate that their work will lead to more reliable applications of LLMs by enabling them to learn input-label relationships more effectively from demonstrations. They also suggest that future work could leverage their analysis to mitigate harmful biases in pre-trained models through demonstration-based methods.
Summary updated