TL;DR
Large Language Models (LLMs) often act more like stubborn experts than flexible students. This paper demonstrates that in subjective tasks like emotion recognition, LLMs frequently ignore the specific examples provided in "In-Context Learning" (ICL) and instead default to their own internal "priors"—pre-existing biases from their training data. Most surprisingly, the larger the model, the stronger this "gravity well" of prior knowledge becomes, making bigger models less adaptable to the nuances of specific datasets.
What Problem Does This Paper Address?
In-Context Learning (ICL) is the "magic" that allows an LLM to perform a new task just by looking at a few examples in a prompt, without any weight updates. The assumption is that the model "learns" the mapping between inputs and outputs from these examples.
However, for subjective tasks like emotion recognition, there is no single "correct" mapping. One dataset might label a tweet as "optimistic," while another calls it "joyful." The authors investigate a critical tension: does the LLM actually learn the specific mapping provided in the prompt (Task Learning), or does it simply recognize the general task and fetch its own pre-existing opinion (Task Recognition)? If the latter is true, ICL might be failing to help models adapt to specialized domains or subjective human annotations.
Methods
The researchers designed experiments to quantify the "pull" and "consistency" of LLM priors across several models (Gemma-7b, OLMo-7B, LLaMA 2 13b/70b, and GPT-3.5/4). Their approach included:
- Task-Recognition Zero-Shot: Prompting models with demonstrations where labels were randomized. If performance remains steady despite "wrong" examples, the model is relying on its internal prior rather than the prompt.
- Quantifying "Pull": They measured how similar a model's final predictions were to the ground truth versus how similar they were to the model's own "zero-shot" (un-prompted) predictions.
- Consistency Analysis: They varied prompt configurations to see if the model's internal "opinion" stayed the same across different ways of asking the question.
- Reinforced Priors: They tested if models could better follow a mapping if the prompt's "ground truth" was replaced with the model's own prior predictions (essentially asking the model to be consistent with itself).
Datasets
The study focused on two major multilabel emotion recognition benchmarks involving social media text:
- SemEval 2018 Task 1 E-c: 11 emotion categories (anger, joy, trust, etc.) annotated in tweets.
- GoEmotions: Originally 27 emotions from Reddit comments, pooled into 7 clusters (admiration, anger, fear, joy, optimism, sadness, and surprise) for this study.
- Baseline: They compared LLM performance against Demux, a specialized BERT-based model.
Key Insights and Ideas
- The "Gravity Well" Metaphor: The authors describe LLM priors as gravity wells in function space. Once a model is large enough, its internal knowledge is so "deep" that few-shot examples (ICL) aren't strong enough to nudge the model out of its pre-existing beliefs.
- The Stubbornness of Scale: Contrary to the usual "bigger is better" trend in AI, larger models (like LLaMA 2 70b and GPT-3.5) were actually less amenable to ICL. They showed a much stronger pull toward their internal priors and almost zero improvement from seeing more examples.
- LLMs vs. Small Supervised Models: Even the most powerful LLMs significantly underperformed a much smaller, specialized BERT model (Demux). In GoEmotions, Demux outperformed LLMs by more than 50%, suggesting that LLMs struggle to capture the specific annotation styles of human datasets.
- Inconsistent Experts: Interestingly, while the pull of the prior is strong, the priors themselves are inconsistent. A model might give different "prior" answers if the prompt template changes slightly, yet it still refuses to follow the provided examples.
Key Results
- Performance Saturation: For GPT-3.5 and LLaMA 2 70b, adding more examples (up to 25-shot) resulted in virtually no performance gain over their own zero-shot priors.
- Prior Pull: In the SemEval dataset, the similarity between the model's predictions and its own internal prior was consistently higher than the similarity to the actual ground truth labels provided in the prompt.
- Self-Consistency Wins: When models were prompted with their own prior predictions as "ground truth," their performance (consistency) nearly doubled. This proves the models can follow a mapping; they just fundamentally disagree with the human-provided ones.
Limitations and Future Work
The authors acknowledge that emotion recognition is inherently difficult due to the lack of context in short social media posts. They also noted that while they standardized prompt templates, LLMs remain highly sensitive to formatting.
Future Directions: * Flexible Fine-tuning: The results suggest a need for fine-tuning methods that make LLMs "less rigid" and more capable of adopting new mappings for subjective tasks. * Disentangling RLHF: Future research could investigate how Reinforcement Learning from Human Feedback (RLHF) contributes to the "ossification" of these priors. * Beyond Emotion: Applying this "pull of the prior" analysis to other subjective fields like ethics, humor, or political alignment.
Summary updated