Discovering and Interpreting Biased Concepts in Online Communities

Xavier Ferrer Aran, Tom van Nuenen, Natalia Criado, Jose M. SuchIEEE TKDE 2023 · OpenReview

TL;DR

This paper introduces a novel data-driven method to automatically discover and interpret subtle linguistic biases within online communities, moving beyond pre-defined stereotypes. By combining word embedding analysis with semantic clustering and novel ranking metrics, the approach reveals nuanced conceptual biases that are often missed by existing methods, offering a more comprehensive understanding of community-specific language.

What Problem Does This Paper Address?

Existing methods for detecting biases in language, particularly within word embeddings, often fall into two camps: either they focus on pre-defined, mainstream stereotypes (like gender-profession associations) or they simply identify individual biased words without providing a conceptual understanding. This is problematic for analyzing online communities, which can develop unique biases not present in broader society. The challenge is to uncover these community-specific biases and understand their meaning at a conceptual level, rather than just identifying isolated words or relying on pre-set notions of what biases should exist.

Methods

The authors propose a two-step approach:

  1. Discovering Biased Concepts:
    • Bias Salience: They introduce a "salience" metric that combines both the strength of a word's bias (how much it's associated with one attribute concept over another, using cosine similarity and bias strength formula) and its frequency in the corpus. This ensures that discovered biases are both significant and representative of the community's language.
  2. Most Salient Words: Words exceeding a statistically determined salience threshold (mean + n standard deviations) are selected.
  3. Semantic Word Clustering: These salient words are then clustered using k-means based on their embedding similarity. This groups semantically related biased words into conceptual clusters.
  4. Interpreting Biased Concepts:
    • Categorizing Biased Concepts: Each cluster is assigned a semantic tag (e.g., "Power," "Relationships," "Crime") using the USAS (UCREL Semantic Analysis System) tagset. The relative frequency of these tags provides a high-level overview of the community's biases.
  5. Ranking Biased Concepts: Clusters are further ranked based on three metrics:
  6. Frequency (Rf): Aggregated frequency of words in the cluster.
  7. Strength (Re): Average bias strength of words in the cluster.
  8. Sentiment (Rs): Average sentiment polarity of words in the cluster (both positive and negative).

This multi-faceted approach allows for both a general understanding of bias categories and a detailed view of how specific concepts are expressed.

Datasets

  • Reddit Communities:
    • /r/TheRedPill: Used to study gender biases.
  • /r/Atheism: Used to study religion biases (Islam vs. Christianity).
  • Google News Pre-trained Model: Used for validation, comparing discovered biases against previously attested, pre-defined biases.

The authors trained their own word embedding models for the Reddit datasets using a skip-gram word2vec model.

Key Insights and Ideas

  • Beyond Predefined Stereotypes: The method successfully uncovers biases that are specific to the online communities studied, rather than just reflecting general societal biases. For example, biases in /r/TheRedPill are more nuanced than just simple gender-profession links.
  • Conceptual Understanding is Crucial: Simply identifying biased words is insufficient. Clustering these words into semantic concepts (e.g., "objectification," "power," "violence") provides a much richer and interpretable understanding of the biases at play.
  • Frequency Matters: Combining bias strength with word frequency is key to identifying salient biases that are truly representative of a community, avoiding noise from rare but strongly biased words.
  • Community-Specific Language: Online communities develop unique vocabularies and conceptual frameworks that reflect their specific norms and realities, and this method can effectively capture these.
  • Stability is Achievable: Despite the inherent instability of word embeddings, the proposed method, particularly through semantic clustering, demonstrates good stability across different model training runs.

Key Results

  • Reddit Analysis:
    • /r/TheRedPill: Discovered biases towards women were heavily related to "relationship intimate/sexual," "appearance," and "objects," indicating objectification. Biases towards men focused on "personal names," "power," "warfare," and "crime," suggesting associations with strength and agency.
  • /r/Atheism: Biases towards Islam were strongly associated with "warfare," "crime," and "names" (often of figures linked to socio-political issues), with a predominantly negative sentiment. Biases towards Christianity were more descriptive, focusing on "religion" itself, with a more neutral to positive sentiment.
  • Google News Validation: The method successfully identified pre-defined gender biases (women with family/arts, men with career/science) that were previously attested. Crucially, it also discovered additional biases, such as those related to physical appearance for women, which were stronger than previously reported biases.
  • Stability: Bootstrapping experiments showed high overlap coefficients (average of 0.83 for women/men tags in /r/TheRedPill) for semantic tags, indicating that the discovered conceptual biases are robust to variations in training data.
  • Parameter Influence: Low embedding dimensionality (d=100) had the most significant impact on discovered biases, while cluster repetitions, window sizes, and epochs had less influence. The salience threshold (n) acts as a "zoom" mechanism.

Limitations and Future Work

  • Human-in-the-Loop: The authors emphasize that their method is automated but designed to be used with human oversight. Interpreting the full context of biased concepts, especially those with positive sentiment that might still be part of a problematic discourse (e.g., casual relationships in /r/TheRedPill), requires human judgment.
  • Target Concept Definition: The choice and definition of attribute concepts (e.g., "gender" as binary women/men) are critical and may need adaptation for more fluid concepts like gender identity.
  • Sentiment Analysis Agnosticism: While the method is agnostic to the sentiment analysis tool used, the quality of the sentiment analysis directly impacts the Rs ranking.
  • Future Work: The authors suggest extending the method to other languages and domains, and further exploring the impact of different embedding models (like BERT). They also highlight the potential for their tool to assist platform administrators in identifying and addressing harmful content.

Summary updated

Abstract

Language carries implicit human biases, functioning both as a reflection and a perpetuation of stereotypes that people carry with them. Recently, ML-based NLP methods such as word embeddings have been shown to learn such language biases with striking accuracy. This capability of word embeddings has been successfully exploited as a tool to quantify and study human biases. However, previous studies only consider a predefined set of biased concepts to attest (e.g., whether gender is more or less associated with particular jobs), or just discover biased words without helping to understand their meaning at the conceptual level. As such, these approaches can be either unable to find biased concepts that have not been defined in advance, or the biases they find are difficult to interpret and study. This could make existing approaches unsuitable to discover and interpret biases in online communities, as such communities may carry different biases than those in mainstream culture. This paper improves upon, extends, and evaluates our previous data-driven method to automatically discover and help interpret biased concepts encoded in word embeddings. We apply this approach to study the biased concepts present in the language used in online communities and experimentally show the validity and stability of our method.
BibTeX
@article{aran2023discovering,
  author = {Xavier Ferrer Aran and Tom van Nuenen and Natalia Criado and Jose M. Such},
  title = {{Discovering and Interpreting Biased Concepts in Online Communities}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2023},
  url = {https://openreview.net/forum?id=YhQ4UXJPS9},
}
Figure