Evaluating Large Language Model Biases in Persona-Steered Generation

Andy Liu, Mona T. Diab, Daniel Fried

TL;DR

This paper investigates how well Large Language Models (LLMs) can generate text reflecting diverse, multifaceted personas, particularly those with "incongruous" traits (where one trait clashes with typical associations of another). They find LLMs struggle with these incongruous personas, often defaulting to stereotypes, and that fine-tuning methods like RLHF, while improving steerability, can reduce diversity.

What Problem Does This Paper Address?

LLMs are increasingly used to simulate human-like text generation, often steered by specific personas. While previous work has explored bias in LLMs through multiple-choice questions or one-dimensional personas, this paper tackles the challenge of evaluating LLMs' ability to generate text for multifaceted personas, especially when those personas contain conflicting traits (e.g., a political liberal who supports increased military spending). The authors argue that if LLMs can't represent these "incongruous" personas well, they risk perpetuating oversimplified and potentially harmful stereotypes of demographic groups.

Methods

The core of the study involves a persona-steered statement generation task. They define personas by combining a demographic (e.g., political liberal) with a stance on an issue (e.g., advocating for a path to citizenship). To identify "incongruous" personas, they leverage Pew survey data to find stances that are statistically less common among individuals identifying with a particular demographic. LLMs are then prompted to generate statements from the perspective of these personas.

To evaluate the generated statements, they employ GPT-4 as an automated evaluator, validating its performance against human crowdworkers. They also introduce auxiliary metrics like Entailment Diversity (EDIV) and Semantic Diversity (SDIV) to measure the range of perspectives and language used in generations.

Datasets

The primary dataset used is the OpinionsQA dataset, which contains polling data from the Pew Research Center's American Trends Panel (ATP). This dataset provides individualized polling data on demographic characteristics and issue-level stances, crucial for constructing multifaceted personas.

Key Insights and Ideas

  • Incongruity is a major hurdle: LLMs significantly struggle to generate text for personas with conflicting traits, often falling back on stereotypical associations.
  • RLHF's double-edged sword: While Reinforcement Learning from Human Feedback (RLHF) improves steerability, it can lead to narrower, less diverse persona representations.
  • Multiple-choice doesn't predict open-ended: A model's performance on multiple-choice survey questions doesn't reliably predict its ability to generate persona-steered text in an open-ended format.
  • GPT-4 as a reliable evaluator: GPT-4 proves to be a strong proxy for human judgment in evaluating persona-steered generation, with high agreement scores.

Key Results

  • LLMs are, on average, 9.7% less steerable towards incongruous personas compared to congruous ones. This difference is most pronounced for political stances.
  • Models fine-tuned with RLHF show significantly higher steerability, particularly towards stances associated with political liberals and women, but at the cost of up to a 58.2% decrease in semantic diversity.
  • LLM responses to multiple-choice survey questions only predict steerability towards those stances about 51.5% of the time, only slightly better than random chance.
  • GPT-4 evaluations show an F1 score of 96.3% and a Cohen's Kappa of 0.808 when compared to human annotations, indicating strong correlation.
  • Generating from incongruous personas reduces demographic exaggeration but leads to a significant decrease in semantic diversity.

Limitations and Future Work

The authors acknowledge that their study simplifies complex political issues into binary stances, which might contribute to stereotypical behavior. They also note that they only evaluate single-statement generation, which differs from how LLM simulations are deployed in interactive downstream tasks. Future work could explore more complex, multifaceted persona representations and investigate how LLM steerability impacts broader social dynamics, potentially leading to increased polarization or entrenching divides. They also emphasize the need for further research into interactive settings to better understand and mitigate these biases.

Summary updated

BibTeX
@inproceedings{liu2024evaluating,
  author = {Andy Liu and Mona T. Diab and Daniel Fried},
  title = {{Evaluating Large Language Model Biases in Persona-Steered Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  url = {https://openreview.net/forum?id=rGJwZSKEPU},
}
Figure