TL;DR
This paper investigates how stable language models' (LMs) political leanings are when faced with paraphrased questions. They found that even models showing high overall stability can be swayed by specific paraphrases, and that LMs tend to agree more with positively phrased statements, suggesting potential biases in how we assess LM political leanings.
What Problem Does This Paper Address?
Researchers are increasingly using psychometric tests, originally designed for humans, to understand the "personalities" and political leanings of large language models (LMs). However, LMs are known to be sensitive to how questions are phrased (prompt design). This sensitivity can lead to inconsistent responses, making it hard to reliably assess their political leanings. Previous studies have often used a small number of paraphrases, limiting the statistical power and generalizability of their findings. This paper aims to address this by conducting a large-scale investigation into LM response stability under paraphrasing, using a significantly larger number of paraphrases than prior work.
Methods
The researchers prompted 18 different LMs (both masked language models like BERT and generative models like GPT) with statements from the Political Compass Test. For each statement, they generated 500 paraphrases using GPT-3.5 (and a subset using Claude-3-5-Sonnet to check for model bias in paraphrase generation). They then extracted the probability of the LM agreeing or disagreeing with each statement.
To assess stability, they used several metrics:
- Response Validity: The proportion of responses that were in the requested binary format (yes/no).
- Overall Model Stability: Measured by the range of responses across all paraphrases, standard deviation, and the proportion of "flip" instances where a majority of paraphrases disagreed with the overall trend.
- Per-Item Stability (within model): How much a model's response to a single statement varies across its paraphrases.
- Per-Item Stability (across models): How consistently items elicit unstable responses across different models.
They also analyzed structural features of the paraphrased statements (like sentence length, word length, sentiment, and syntactic dependencies) to see if these influenced model responses.
Datasets
The primary dataset used was the set of statements from the Political Compass Test. The authors generated 500 paraphrases for each of the 62 statements in this test. No new benchmark datasets were introduced, but the scale of paraphrasing for an existing benchmark is a key contribution.
Key Insights and Ideas
- Paraphrasing is a powerful disruptor: Even models that appear stable on average can be significantly swayed by specific paraphrases, highlighting the fragility of LM responses.
- Sentiment matters: LMs show a strong tendency to agree with statements that have a positive sentiment, suggesting a potential "acquiescence bias" similar to humans, but possibly amplified.
- Item-level instability is common: While overall model stability might look good, individual statements can be highly unstable across paraphrases, making them poor indicators of a stable political leaning.
- Not all LMs are created equal: Generative LMs (GLMs) generally showed lower response validity compared to masked LMs (MLMs), and even within GLMs, there were significant differences.
Key Results
- Response Validity: MLMs and GPT-3.5/4/40 showed high response validity (close to 1.00), meaning they consistently provided "yes" or "no" answers. Other GLMs had lower validity.
- Overall Stability: While many models had moderate to high stability in terms of standard deviation and flip proportions, they also exhibited large min-max ranges, indicating that extreme responses were possible.
- Per-Item Stability: Response stability varied considerably within models for different statements. Some statements, like quotations, were consistently unstable across all models, suggesting they might not be suitable for assessing political leanings.
- Structural Predictors: Positive sentiment was consistently associated with a higher probability of agreement across most models. Longer sentences also tended to increase agreement for most models. Syntactic features showed more varied effects depending on the model type.
Limitations and Future Work
The authors acknowledge that their experiments used a single prompt template for most models, and prompt formatting can significantly impact LM responses. They also note that collapsing the four response options of the original Political Compass test into two (agree/disagree) might affect the assessment of political positions.
Future work could explore:
- Designing instruments specifically for LMs that control for structural biases and varying item stability.
- Investigating how to stabilize LM responses, perhaps by providing more in-context examples.
- Conducting group-level assessments with a large number of conditions for each item to get a more robust understanding of LM properties.
- Further detailed investigations into how structural features affect model responses.
Summary updated