Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity

Gabriel SimmonsACL 2023

TL;DR

This paper investigates whether Large Language Models (LLMs) can mimic the moral biases associated with political identities, a phenomenon termed "moral mimicry." The study finds that LLMs do indeed reproduce these biases, generating moral rationalizations that align with either liberal or conservative viewpoints when prompted. This mimicry is influenced by model size and suggests LLMs can be steered to adopt specific ideological stances.

What Problem Does This Paper Address?

As LLMs become more sophisticated and integrated into our information ecosystem, understanding their potential to influence human opinions and spread disinformation is crucial. Morality plays a significant role in persuasion and polarization. This paper addresses the question of whether LLMs can replicate the moral biases observed in human political groups, specifically within the United States. This capability, termed "moral mimicry," is important because it could allow LLMs to generate content that resonates with specific political identities, potentially exacerbating societal divides or, conversely, facilitating dialogue. The study aims to characterize this mimicry and explore its relationship with model size and human moral language use.

Methods

The core of the methodology involves constructing prompts that guide LLMs to generate moral rationalizations. These prompts combine three key variables:

  1. Scenarios: Text descriptions of situations or actions requiring moral judgment.
  2. Political Identity Phrases: Text strings indicating a political stance (e.g., "liberal," "conservative").
  3. Moral Stances: Indications of whether the scenario is to be judged as moral or immoral.

These components are fed into various LLMs using different prompt templates. The generated text completions are then analyzed using Moral Foundations Theory (MFT). MFT posits that human morality is based on five foundations: Care/Harm, Fairness/Cheating, Loyalty/Betrayal, Authority/Subversion, and Sanctity/Degradation. The study uses Moral Foundations Dictionaries (MFDv1, MFDv2, and eMFD) to quantify the presence of these moral foundations in the LLM-generated text.

To measure "moral mimicry," the researchers calculated effect sizes representing the difference in foundation expression when a liberal versus a conservative identity was prompted, keeping the scenario and stance constant. This allowed them to quantify how much the LLM's moral language shifted based on the political identity cue.

Datasets

The study utilized several datasets for scenarios:

  • Moral Stories: Provided scenarios for moral and immoral situations.
  • ETHICS: A dataset containing scenarios related to ethical frameworks, from which "short" examples were used.
  • Social Chemistry 101: This dataset was used in two ways:
  • Social Chemistry Actions: Provided scenarios for analyzing LLM responses to specific actions.
  • Social Chemistry Situations: Used for comparing LLM outputs to human consensus on moral reasoning.

The Moral Foundations Dictionaries (MFDv1, MFDv2, and eMFD) were used for analyzing the moral content of the generated text.

Key Insights and Ideas

  • LLMs are Moral Mimics: The central finding is that LLMs can indeed reproduce moral biases associated with political identities. When prompted with "liberal" or "conservative" identities, the models generate text that reflects the corresponding moral leanings.
  • Situational Appropriateness: LLMs show a degree of situational awareness, increasing their use of moral foundations that are salient to a given scenario. However, their overall moral language use still differs more from human consensus than human-human differences do.
  • Model Size Matters (Somewhat): Moral mimicry generally increases with model size, particularly within the OPT family. However, size isn't the sole determinant; the GPT-3.5 models, which underwent additional supervised fine-tuning and pre-training on text and code, exhibited stronger moral mimicry than some larger GPT-3 models.
  • Political Identity Shapes Moral Language: The study demonstrates that explicitly conditioning LLMs with political identities leads to distinct patterns of moral foundation use, aligning with established findings from Moral Foundations Theory about human political groups.

Key Results

  • Criterion A Met: LLMs were found to increase their use of foundation-specific words when those foundations were salient in a scenario, meeting a weaker criterion for appropriate moral language use.
  • Criterion B Not Met: LLMs exhibited greater differences from human consensus moral language use than humans do among themselves.
  • Moral Foundations Hypothesis Supported: For most LLMs tested, prompting with a "liberal" identity led to increased use of individualizing foundations (Care/Harm, Fairness/Cheating), while prompting with a "conservative" identity led to increased use of binding foundations (Loyalty/Betrayal, Authority/Subversion, Sanctity/Degradation).
  • Model Size Correlation: For the OPT family, there was a moderate positive correlation (r=0.69) between model size and the strength of moral mimicry. When GPT-3 and GPT-3.5 models were analyzed together, this correlation increased significantly (r=0.84).
  • GPT-3.5 Superiority: GPT-3.5 models, particularly text-davinci-002 and text-davinci-003, showed the strongest moral mimicry capabilities.

Limitations and Future Work

  • Dictionary Limitations: The study relies on moral foundations dictionaries, which may not capture the full nuance of human moral expression. Future work could incorporate machine learning approaches or human evaluations for a more comprehensive analysis.
  • Prompt Sensitivity: While efforts were made to control for prompt phrasing by using multiple templates, the choice of prompts was made by the author, suggesting potential for more principled prompt selection in future studies.
  • Demographic Bias: The human studies referenced were primarily conducted on US populations, and political connotations can vary across demographics. Future research could explore multilingual models and diverse demographic groups.
  • Dataset Bias: The datasets used for training and annotation may reflect biases of the crowd workers who contributed to them, potentially leaning towards certain political viewpoints or moral foundations.
  • Model Availability: The proprietary nature of GPT-3 and GPT-3.5 model weights limits direct experimentation with these models, posing a barrier for future research compared to openly available models.
  • Future Directions: The authors suggest exploring how training data and practices influence moral mimicry, investigating how to influence this mimicry, and applying similar methods to other theories of morality. They also highlight the need to understand how LLMs' ability to simulate political identities might impact end-users and society.

Summary updated

BibTeX
@inproceedings{simmons2023moral,
  author = {Gabriel Simmons},
  title = {{Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure