Gender and Representation Bias in GPT-3 Generated Stories

Li Lucy, David BammanNAACL 2021

TL;DR

This paper investigates gender stereotypes in stories generated by GPT-3, finding that the model often reproduces societal biases even when prompts lack explicit gender cues. By analyzing topic distributions and descriptive language, the authors demonstrate that GPT-3 associates different traits and themes with characters based on their perceived gender, mirroring biases found in human-authored literature.

What Problem Does This Paper Address?

Large language models (LLMs) like GPT-3 are increasingly used for creative tasks, including storytelling. However, these models are trained on vast amounts of text data that inherently contain societal biases. This raises a critical concern: LLMs might not only reflect but also amplify these biases, leading to the generation of narratives that reinforce harmful stereotypes. This paper specifically addresses the problem of how gender stereotypes manifest in GPT-3 generated stories and whether these biases can be mitigated through prompt engineering. The authors aim to understand the nature and extent of these representational harms, particularly how GPT-3 associates different topics and character descriptions with perceived gender.

Methods

The researchers employed a multi-pronged approach to analyze GPT-3's generated stories:

  • Topic Modeling: They used Latent Dirichlet Allocation (LDA) to identify recurring themes within both GPT-3 generated stories and human-authored book excerpts. This allowed them to compare the thematic content associated with characters of different genders.
  • Lexicon-Based Stereotype Measurement: To quantify character descriptions, they developed lexicons for "appearance," "intellect," and "power." They then used word embeddings (word2vec) and semantic similarity measures to assess how frequently adjectives and verbs related to these categories were associated with characters of different genders. For "power," they specifically used a semantic axis approach to differentiate between strength/dominance and weakness/dependence.
  • Gender Inference: A crucial step was inferring the gender of characters. This was primarily done by analyzing pronoun usage (he/him/his vs. she/her/hers). For characters without clear pronouns, they used name-based gender prediction, leveraging US birth name data.
  • Matched Stories Analysis: To isolate the effect of gender, they created a subset of stories where GPT-3 generated multiple narratives from the exact same prompt, allowing for direct comparison of outputs when only the inferred gender of the main character differed.

Datasets

  • Prompts: Single sentences containing main characters, sampled from 402 contemporary English fiction books. These books were sourced from various collections, including the Black Book Interactive Project and Pulitzer Prize winners.
  • Book Excerpts: Human-authored text that began with the same sentences used as prompts for GPT-3, serving as a control.
  • GPT-3 Generated Stories: Text generated by the GPT-3 API (davinci model) using the sampled prompts.

The paper does not introduce a new benchmark dataset but rather uses existing literary data and LLM outputs for analysis.

Key Insights and Ideas

  • GPT-3 Mimics Literary Biases: The study found that GPT-3's generated stories exhibit gender stereotypes that are remarkably similar to those present in human-authored literature. This suggests that the model is effectively learning and reproducing patterns from its training data.
  • Implicit Bias is Pervasive: Even when prompts intentionally avoided explicit gender cues, GPT-3 still assigned different topics and descriptions based on the inferred gender of the character. This highlights the deep-seated nature of these biases within the model.
  • Stereotypes Persist: If a prompt contained a word related to a stereotype (e.g., "family" for feminine characters), the generated story was more likely to amplify that stereotype if the character's gender aligned with it.
  • Prompting for Intellect Works, Power is Tricky: The researchers found that using prompts with cognitive verbs could steer GPT-3 towards generating characters with higher intellect scores. However, prompts with high-power verbs did not significantly increase the power attributed to non-masculine characters, suggesting that traditional power differentials are harder to overcome.

Key Results

  • Topic Differences: Feminine characters were more frequently associated with topics related to "family," "emotions," and "body parts," while masculine characters were linked to "politics," "war," "sports," and "crime." These differences were statistically significant and mirrored those found in books.
  • Descriptive Differences: Feminine characters were more likely to be described in terms of "appearance," while masculine characters were described as more "powerful." These differences persisted even in matched GPT-3 stories.
  • Character Counts: Both books and GPT-3 generated stories tend to feature more masculine characters than feminine ones.
  • Intellect vs. Power: While prompts with cognitive verbs increased intellect scores, prompts with high-power verbs did not lead to a comparable increase in power for non-masculine characters, indicating a persistent gender gap in power attribution.

Limitations and Future Work

  • Gender Inference Limitations: The authors acknowledge that their pronoun-based gender inference method has limitations, including potential coreference errors, inability to capture gender fluidity, and exclusion of nonbinary genders. Name-based inference also has limitations regarding cultural specificity and temporal changes.
  • Focus on Conceptual Gender: The study focuses on inferring conceptual gender (how a perceiver might view gender) rather than an individual's self-determined identity, which is a necessary simplification for analyzing fictional text but has ethical implications.
  • Mitigation Challenges: While prompt design can influence output, the authors emphasize that it's not a complete solution and doesn't shift responsibility from system creators.
  • Future Directions: The authors suggest future work could explore causal inference and more sophisticated prompt design to better understand and mitigate the factors influencing GPT-3's narrative outputs and their biases. They also note the need for improved methods for respectful measurement of gender in fiction.

Summary updated

BibTeX
@inproceedings{lucy2021gender,
  author = {Li Lucy and David Bamman},
  title = {{Gender and Representation Bias in GPT-3 Generated Stories}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure