The Strong Pull of Prior Knowledge in Large Language Models and Its Impact on Emotion Recognition

Georgios Chochlakis, Alexandros Potamianos, Kristina Lerman, Shrikanth Narayanan

Abstract

In-context Learning (ICL) has emerged as a powerful paradigm for performing natural language tasks with Large Language Models (LLM) without updating the models' parameters, in contrast to the traditional gradient-based finetuning. The promise of ICL is that the LLM can adapt to perform the present task at a competitive or state-of-the-art level at a fraction of the cost. The ability of LLMs to perform tasks in this few-shot manner relies on their background knowledge of the task (or task priors). However, recent work has found that, unlike traditional learning, LLMs are unable to fully integrate information from demonstrations that contrast task priors. This can lead to performance saturation at suboptimal levels, especially for subjective tasks such as emotion recognition, where the mapping from text to emotions can differ widely due to variability in human annotations. In this work, we design experiments and propose measurements to explicitly quantify the consistency of proxies of LLM priors and their pull on the posteriors. We show that LLMs have strong yet inconsistent priors in emotion recognition that ossify their predictions. We also find that the larger the model, the stronger these effects become. Our results suggest that caution is needed when using ICL with larger LLMs for affect-centered tasks outside their pre-training domain and when interpreting ICL results.

Introduction

In-Context Learning (ICL) is often viewed as a mechanism for models to "learn" new mappings between inputs and outputs on the fly. By providing a few examples in a prompt, researchers expect LLMs to adapt to the specific nuances of a dataset. However, for subjective tasks like emotion recognition, where labels like "joy" or "optimism" can be interpreted differently across datasets, a critical tension arises: does the LLM actually learn the provided mapping, or does it simply recognize the task and default to its own internal "priors"?

This paper investigates whether LLMs act as flexible students or stubborn experts. If models rely too heavily on pre-existing biases from their training data, ICL may fail to help them adapt to specialized domains or specific human annotation styles.

Methodology

To quantify the "pull" and "consistency" of LLM priors, the researchers evaluated several models, including Gemma-7b, OLMo-7B, LLaMA 2 (13b and 70b), and GPT-3.5/4. The experimental design included:

  • Task-Recognition Zero-Shot: Testing models with randomized labels in demonstrations. If performance remains steady despite incorrect examples, the model is likely relying on its internal prior.
  • Quantifying "Pull": Measuring the similarity between a model's final predictions and the ground truth versus its similarity to its own un-prompted (zero-shot) predictions.
  • Consistency Analysis: Varying prompt configurations to determine if the model's internal "opinion" remains stable across different templates.
  • Reinforced Priors: Prompting models with their own prior predictions as "ground truth" to see if they can follow a mapping they already agree with.

Experiments were conducted on two major benchmarks: SemEval 2018 Task 1 E-c (11 emotion categories) and GoEmotions (pooled into 7 clusters). Performance was compared against Demux, a specialized BERT-based model.

The Gravity Well of Priors

The authors introduce the metaphor of "gravity wells" to describe LLM priors in function space. Once a model reaches a certain scale, its internal knowledge becomes so deep that few-shot examples are insufficient to nudge the model toward a different ground-truth mapping.

Conceptual visualization of LLM priors as gravity wells that resist the pull of in-context learning toward ground-truth mappings.

Contrary to the typical "bigger is better" trend, larger models like LLaMA 2 70b and GPT-3.5 were found to be less amenable to ICL. These models exhibited a much stronger pull toward their internal priors and showed almost zero improvement as the number of examples increased. Interestingly, while these priors are strong, they are also inconsistent; a model might provide different "prior" answers if the prompt template changes slightly, yet it still refuses to follow the provided examples in the prompt.

Experimental Results

The empirical results highlight a significant gap between LLMs and specialized supervised models. In the GoEmotions dataset, the BERT-based Demux model outperformed LLMs by more than 50%, suggesting that LLMs struggle to capture the specific annotation styles of human datasets.

Improvement over best task prior across different models and shot counts, showing performance saturation in larger models.

Key findings from the performance analysis include:

  • Performance Saturation: For larger models like GPT-3.5 and LLaMA 2 70b, increasing the number of examples up to 25-shot resulted in virtually no performance gain over their zero-shot priors.
  • Prior Pull: In the SemEval dataset, the similarity between model predictions and their internal priors was consistently higher than the similarity to the actual ground truth labels provided in the prompt.
  • Self-Consistency: When models were prompted with their own prior predictions as the "ground truth," their consistency nearly doubled. This indicates that models can follow a mapping; they simply fundamentally disagree with the human-provided ones in the dataset.

Conclusion

The findings suggest that the "ossification" of priors in large LLMs presents a significant hurdle for subjective tasks. Future work should focus on developing flexible fine-tuning methods that allow LLMs to adopt new mappings more easily. Additionally, investigating how Reinforcement Learning from Human Feedback (RLHF) contributes to the rigidity of these priors could provide insights into making models more adaptable for affect-centered tasks outside their initial training domain.

Summary updated

Abstract

In-context Learning (ICL) has emerged as a powerful paradigm for performing natural language tasks with Large Language Models (LLM) without updating the models' parameters, in contrast to the traditional gradient-based finetuning. The promise of ICL is that the LLM can adapt to perform the present task at a competitive or state-of-the-art level at a fraction of the cost. The ability of LLMs to perform tasks in this few-shot manner relies on their background knowledge of the task (or task priors). However, recent work has found that, unlike traditional learning, LLMs are unable to fully integrate information from demonstrations that contrast task priors. This can lead to performance saturation at suboptimal levels, especially for subjective tasks such as emotion recognition, where the mapping from text to emotions can differ widely due to variability in human annotations. In this work, we design experiments and propose measurements to explicitly quantify the consistency of proxies of LLM priors and their pull on the posteriors. We show that LLMs have strong yet inconsistent priors in emotion recognition that ossify their predictions. We also find that the larger the model, the stronger these effects become. Our results suggest that caution is needed when using ICL with larger LLMs for affect-centered tasks outside their pre-training domain and when interpreting ICL results.
BibTeX
@inproceedings{chochlakis2024strong,
  author = {Georgios Chochlakis and Alexandros Potamianos and Kristina Lerman and Shrikanth Narayanan},
  title = {{The Strong Pull of Prior Knowledge in Large Language Models and Its Impact on Emotion Recognition}},
  booktitle = {ACII},
  year = {2024},
  url = {https://openreview.net/forum?id=L6IiiYMGjr},
}

Loading related content...

Figure