The Strong Pull of Prior Knowledge in Large Language Models and Its Impact on Emotion Recognition

Georgios Chochlakis, Alexandros Potamianos, Kristina Lerman, Shrikanth Narayanan

TL;DR

Large Language Models (LLMs) often act more like stubborn experts than flexible students. This paper demonstrates that in subjective tasks like emotion recognition, LLMs frequently ignore the specific examples provided in "In-Context Learning" (ICL) and instead default to their own internal "priors"—pre-existing biases from their training data. Most surprisingly, the larger the model, the stronger this "gravity well" of prior knowledge becomes, making bigger models less adaptable to the nuances of specific datasets.

What Problem Does This Paper Address?

In-Context Learning (ICL) is the "magic" that allows an LLM to perform a new task just by looking at a few examples in a prompt, without any weight updates. The assumption is that the model "learns" the mapping between inputs and outputs from these examples.

However, for subjective tasks like emotion recognition, there is no single "correct" mapping. One dataset might label a tweet as "optimistic," while another calls it "joyful." The authors investigate a critical tension: does the LLM actually learn the specific mapping provided in the prompt (Task Learning), or does it simply recognize the general task and fetch its own pre-existing opinion (Task Recognition)? If the latter is true, ICL might be failing to help models adapt to specialized domains or subjective human annotations.

Methods

The researchers designed experiments to quantify the "pull" and "consistency" of LLM priors across several models (Gemma-7b, OLMo-7B, LLaMA 2 13b/70b, and GPT-3.5/4). Their approach included:

  • Task-Recognition Zero-Shot: Prompting models with demonstrations where labels were randomized. If performance remains steady despite "wrong" examples, the model is relying on its internal prior rather than the prompt.
  • Quantifying "Pull": They measured how similar a model's final predictions were to the ground truth versus how similar they were to the model's own "zero-shot" (un-prompted) predictions.
  • Consistency Analysis: They varied prompt configurations to see if the model's internal "opinion" stayed the same across different ways of asking the question.
  • Reinforced Priors: They tested if models could better follow a mapping if the prompt's "ground truth" was replaced with the model's own prior predictions (essentially asking the model to be consistent with itself).

Datasets

The study focused on two major multilabel emotion recognition benchmarks involving social media text:

  1. SemEval 2018 Task 1 E-c: 11 emotion categories (anger, joy, trust, etc.) annotated in tweets.
  2. GoEmotions: Originally 27 emotions from Reddit comments, pooled into 7 clusters (admiration, anger, fear, joy, optimism, sadness, and surprise) for this study.
  3. Baseline: They compared LLM performance against Demux, a specialized BERT-based model.

Key Insights and Ideas

  • The "Gravity Well" Metaphor: The authors describe LLM priors as gravity wells in function space. Once a model is large enough, its internal knowledge is so "deep" that few-shot examples (ICL) aren't strong enough to nudge the model out of its pre-existing beliefs.
  • The Stubbornness of Scale: Contrary to the usual "bigger is better" trend in AI, larger models (like LLaMA 2 70b and GPT-3.5) were actually less amenable to ICL. They showed a much stronger pull toward their internal priors and almost zero improvement from seeing more examples.
  • LLMs vs. Small Supervised Models: Even the most powerful LLMs significantly underperformed a much smaller, specialized BERT model (Demux). In GoEmotions, Demux outperformed LLMs by more than 50%, suggesting that LLMs struggle to capture the specific annotation styles of human datasets.
  • Inconsistent Experts: Interestingly, while the pull of the prior is strong, the priors themselves are inconsistent. A model might give different "prior" answers if the prompt template changes slightly, yet it still refuses to follow the provided examples.

Key Results

  • Performance Saturation: For GPT-3.5 and LLaMA 2 70b, adding more examples (up to 25-shot) resulted in virtually no performance gain over their own zero-shot priors.
  • Prior Pull: In the SemEval dataset, the similarity between the model's predictions and its own internal prior was consistently higher than the similarity to the actual ground truth labels provided in the prompt.
  • Self-Consistency Wins: When models were prompted with their own prior predictions as "ground truth," their performance (consistency) nearly doubled. This proves the models can follow a mapping; they just fundamentally disagree with the human-provided ones.

Limitations and Future Work

The authors acknowledge that emotion recognition is inherently difficult due to the lack of context in short social media posts. They also noted that while they standardized prompt templates, LLMs remain highly sensitive to formatting.

Future Directions: * Flexible Fine-tuning: The results suggest a need for fine-tuning methods that make LLMs "less rigid" and more capable of adopting new mappings for subjective tasks. * Disentangling RLHF: Future research could investigate how Reinforcement Learning from Human Feedback (RLHF) contributes to the "ossification" of these priors. * Beyond Emotion: Applying this "pull of the prior" analysis to other subjective fields like ethics, humor, or political alignment.

Summary updated

Abstract

In-context Learning (ICL) has emerged as a power-ful paradigm for performing natural language tasks with Large Language Models (LLM) without updating the models' parameters, in contrast to the traditional gradient-based finetuning. The promise of ICL is that the LLM can adapt to perform the present task at a competitive or state-of-the-art level at a fraction of the cost. The ability of LLMs to perform tasks in this few-shot manner relies on their background knowledge of the task (or task priors). However, recent work has found that, unlike traditional learning, LLMs are unable to fully integrate information from demonstrations that contrast task priors. This can lead to performance saturation at suboptimal levels, especially for subjective tasks such as emotion recognition, where the mapping from text to emotions can differ widely due to variability in human annotations. In this work, we design experiments and propose measurements to explicitly quantify the consistency of proxies of LLM priors and their pull on the posteriors. We show that LLMs have strong yet inconsistent priors in emotion recognition that ossify their predictions. We also find that the larger the model, the stronger these effects become. Our results suggest that caution is needed when using ICL with larger LLMs for affect-centered tasks outside their pretraining domain and when interpreting ICL results.1
BibTeX
@inproceedings{chochlakis2024strong,
  author = {Georgios Chochlakis and Alexandros Potamianos and Kristina Lerman and Shrikanth Narayanan},
  title = {{The Strong Pull of Prior Knowledge in Large Language Models and Its Impact on Emotion Recognition}},
  booktitle = {ACII},
  year = {2024},
  url = {https://openreview.net/forum?id=L6IiiYMGjr},
}

Loading related content...

Figure