What Makes a Good Counter-Stereotype? Evaluating Strategies for Automated Responses to Stereotypical Text

Kathleen Fraser, Svetlana Kiritchenko, Isar Nejadgholi, Anna KerkhofACL 2023

TL;DR

This paper explores how to automatically generate effective "counter-stereotypes" – responses that challenge harmful societal stereotypes online. By prompting ChatGPT with various strategies derived from psychology and NLP research, the authors identify "denouncing," "warning of consequences," and "empathy" as the most promising approaches for future development.

What Problem Does This Paper Address?

Online platforms are rife with stereotypes, which can cause psychological distress to targeted groups and perpetuate discrimination. While detecting stereotypes is an active area of NLP research, the question of how to respond to them remains largely open. Unlike hate speech, stereotypes often don't meet deletion criteria. This paper tackles the challenge of automatically generating responses that can effectively counter these stereotypes without causing offense or introducing new harmful generalizations, aiming to educate both the original poster and bystanders.

Methods

The core of this research involves a two-stage process:

  1. Strategy Identification and Generation: The authors synthesized literature from social psychology and NLP to identify 11 potential counter-stereotype strategies. These include denouncing stereotypes, presenting counter-facts, using humor, warning of consequences, empathy, critical questions, counter-examples, broadening exceptions/universals, emphasizing positive qualities, and perspective-taking. They then used ChatGPT to automatically generate counter-stereotypes for 18 selected stereotypes, employing each of the 11 strategies. Prompts were carefully crafted to elicit single-sentence, tweet-style responses.
  2. Manual Evaluation: Four annotators (the authors) evaluated the generated counter-stereotypes based on two criteria:
  3. Technical Quality: Did ChatGPT correctly use the requested strategy? Was the response inoffensive? Was it believable?
  4. Social Effectiveness: Would the generated response be effective from a bystander's perspective?

Datasets

  • Stereotypes: The study used 18 stereotypes categorized along three dimensions: descriptive vs. prescriptive, positive vs. negative, and statistically accurate vs. inaccurate. These stereotypes were compiled from literature and popular press.
  • Generated Counter-Stereotypes: The paper generated a dataset of counter-stereotypes using ChatGPT based on the 18 stereotypes and 11 strategies. This dataset, along with the prompts used, is detailed in the appendices.

Key Insights and Ideas

  • Not all strategies are created equal: While ChatGPT can generate responses for most strategies, some are significantly more prone to errors or less effective. Strategies like "humor" and "broadening exceptions" proved particularly problematic.
  • The "bystander effect" is key: The evaluation focused on effectiveness from a bystander's viewpoint, acknowledging that directly changing the original speaker's mind is difficult. This shifts the focus to influencing the broader audience.
  • Empathy is powerful, but nuanced: Empathy was rated highly, but often worked best when combined with other strategies. Overly empathetic responses to highly offensive views could be problematic.
  • Denouncing and consequences are robust: Directly calling out a statement as a stereotype and explaining its real-world impact emerged as strong, generalizable strategies.
  • Prescriptive stereotypes are tricky: Stereotypes about how people should be (prescriptive) are harder to counter with facts and may benefit more from critical questions or perspective-taking.
  • Accuracy matters: While "believability" was a primary evaluation criterion, the paper highlights the danger of misinformation. Even believable "facts" can be inaccurate or misleading, underscoring the need for reliable sources.

Key Results

  • Promising Strategies: "Denouncing stereotypes," "warning of consequences," and "empathy" were identified as the most promising strategies, receiving high effectiveness scores.
  • Problematic Strategies: "Humor" had a high rate of being potentially offensive, and "broadening exceptions" was frequently generated incorrectly. "Counter-facts" and "counter-examples" also showed issues with accuracy and effectiveness, especially for highly inaccurate stereotypes.
  • Effectiveness Scores: Figure 2 shows overall effectiveness scores, with denouncing, consequences, and empathy scoring highest. Humor and counter-examples scored lowest.
  • Accuracy of Generated Content: Approximately 40% of the "facts" generated by ChatGPT were found to be incorrect or unverifiable.

Limitations and Future Work

  • Limited Scope: The study focused on explicitly stated stereotypes and did not address implicit or subtle expressions.
  • AI Limitations: ChatGPT's tendency to generate repetitive or generic responses, its inability to cite sources, and its potential for misinformation are significant limitations. The study also noted that ChatGPT claiming cultural identities was problematic.
  • Prompt Sensitivity: The effectiveness of generated responses was influenced by prompt phrasing.
  • Small Annotator Pool: The evaluation was conducted by a small group of authors, limiting the generalizability of findings. Future work should involve a broader, more representative user study.
  • Ambivalent Stereotypes: The study did not deeply explore strategies for ambivalent stereotypes (e.g., positive on one dimension, negative on another).
  • Combining Strategies: The paper suggests that combining multiple strategies in a single response could be more effective and warrants further investigation. Future work should also explore varying ChatGPT's temperature parameter and testing other large language models.

Summary updated

BibTeX
@inproceedings{fraser2023what,
  author = {Kathleen Fraser and Svetlana Kiritchenko and Isar Nejadgholi and Anna Kerkhof},
  title = {{What Makes a Good Counter-Stereotype? Evaluating Strategies for Automated Responses to Stereotypical Text}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure