PRISM: A Methodology for Auditing Biases in Large Language Models

Leif Azzopardi, Yashar MoshfeghiarXiv 2024 · arXiv

TL;DR

PRISM (Preference Revelation through Indirect Stimulus Methodology) is a novel auditing framework that bypasses LLM safety guardrails by using indirect, essay-writing tasks instead of direct multiple-choice questions. By evaluating these essays across various assigned personas, PRISM successfully maps out both the default biases of LLMs and the boundaries of what they are willing to express, drastically reducing model refusal rates.

What Problem Does This Paper Address?

Auditing Large Language Models (LLMs) for biases—whether political, social, or religious—is becoming increasingly difficult. As AI developers implement stricter safety guardrails, fine-tuning, and red-teaming protocols, models are trained to hide or obfuscate their preferences.

When subjected to standard survey instruments like the Political Compass Test, modern LLMs often trigger safety guardrails, resulting in canned neutral responses ("As an AI, I do not have personal opinions") or outright refusals to answer. This makes direct auditing methods highly inefficient, unreliable, and brittle. The authors introduce PRISM to shift the paradigm from direct coercion to indirect, task-based elicitation.

Methods

Instead of forcing an LLM to choose from a rigid Likert scale (e.g., "Strongly Agree" to "Strongly Disagree"), PRISM asks the model to perform a natural generation task: writing a short essay discussing a specific proposition.

The PRISM architecture follows these key steps:

  1. Formulate Prompts: The model is given a statement from a survey instrument and instructed to write an essay arguing for or against it.
  2. Role Priming: To map out the model's behavioral boundaries, the prompt can optionally assign a specific persona or role (e.g., "You are a left-wing liberal..."). Leaving the role blank reveals the model's default stance.
  3. Automated Stance Detection: The generated essay is analyzed by an external assessor (an LLM or human) to determine its level of agreement with the original statement.
  4. Score Tallying: The ratings are aggregated to compute the model's coordinates on the survey's ideological spectrum.
Figure 1: PRISM – A Reliable Methodology for Au- diting LLMs: The LLM is prompted to write an essay for each statement (given a role or not). These essays are rated in terms of their level of agreement with the statement and then used to tally up a score to determine the LLM position given the role it has been assigned (or its default position).

Datasets

The methodology was evaluated using the Political Compass Test (PCT), a popular 62-proposition benchmark that measures political ideology across two axes: economic (Left vs. Right) and social (Libertarian vs. Authoritarian).

The authors audited 21 distinct LLMs from 7 major providers, including OpenAI (GPT-3.5/4 series), Anthropic (Claude 2.1/3 series), Google (Gemini 1.0/1.5), Meta (Llama 2/3 series), Mistral, Cohere, and Alibaba (Qwen).

Key Insights and Ideas

  • Guardrails are Task-Dependent: Models that flatly refuse to answer a direct political question will happily write a nuanced essay defending or attacking the exact same viewpoint. Indirect prompting successfully circumvents defensive alignment.
  • The "Window of Expression": By testing models under extreme roles, PRISM maps out a model's ideological flexibility. Some models (like Llama 2) are highly constrained and struggle to deviate from their default programming, while others (like GPT-4o) possess a broad window of expression and can convincingly argue from many perspectives.
  • Dimensional Conflation (Stereotyping): When forced into specific roles, many LLMs exhibit a form of political stereotyping. For example, if instructed to take an economically right-wing stance, they frequently default to socially authoritarian arguments as well, failing to treat the two axes independently.

Key Results

  • Massive Reduction in Refusals: While direct multiple-choice prompting yielded an average refusal rate of 13% (spiking to a massive 83% for Claude 2.1), PRISM reduced the average refusal rate to just 1%.
  • Clearer Stance Detection: PRISM resulted in fewer ambiguous "neutral" ratings (6% vs. 9% in direct prompting), offering a more precise look at the models' true underlying leanings.
  • Consistent Left-Liberal Default: Confirming prior literature, the unprimed, default positions of almost all audited LLMs clustered firmly within the economically left-leaning and socially liberal quadrant.
Figure 3: The default (no role) position of each LLM. Most LLMs, by default, espouse left and liberal-leaning positions. Mistal.AI’s Mistral model was the most left and liberal, while Cohere’s Command-light model was the most right and authoritarian leaning.

Limitations and Future Work

The authors note several limitations that pave the way for future research:

  • Assessor Bias: PRISM relies on an LLM (GPT-3.5 Turbo in this study) to grade the generated essays. Even though it demonstrated strong alignment with human annotators (88.6% agreement, Cohen's Kappa of 0.774), the automated judge could introduce its own subtle biases.
  • Prompt Engineering Sensitivity: The study utilized a simplified prompting strategy. More complex, real-world prompting techniques might yield different boundaries.
  • Evaluation Scope: The framework was only demonstrated on political bias using the PCT. Future work is required to apply PRISM to other domains, such as auditing gender, racial, or religious biases.

Summary updated

Abstract

Auditing Large Language Models (LLMs) to discover their biases and preferences is an emerging challenge in creating Responsible Artificial Intelligence (AI). While various methods have been proposed to elicit the preferences of such models, countermeasures have been taken by LLM trainers, such that LLMs hide, obfuscate or point blank refuse to disclosure their positions on certain subjects. This paper presents PRISM, a flexible, inquiry-based methodology for auditing LLMs - that seeks to illicit such positions indirectly through task-based inquiry prompting rather than direct inquiry of said preferences. To demonstrate the utility of the methodology, we applied PRISM on the Political Compass Test, where we assessed the political leanings of twenty-one LLMs from seven providers. We show LLMs, by default, espouse positions that are economically left and socially liberal (consistent with prior work). We also show the space of positions that these models are willing to espouse - where some models are more constrained and less compliant than others - while others are more neutral and objective. In sum, PRISM can more reliably probe and audit LLMs to understand their preferences, biases and constraints.
BibTeX
@misc{azzopardi2024prism,
  author = {Leif Azzopardi and Yashar Moshfeghi},
  title = {{PRISM: A Methodology for Auditing Biases in Large Language Models}},
  year = {2024},
  eprint = {2410.18906},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.18906},
}

Loading related content...

Figure