TL;DR
This paper introduces a novel task, Gender-biased Language Identification and Extraction (GLIdE), and a multi-task deep framework to automatically detect gender-biased content and identify its causal phrases in social media. By incorporating emotional information and a gender-stereotyped knowledge graph, their GLIDE model significantly outperforms existing methods, demonstrating a crucial step towards promoting gender equality online.
What Problem Does This Paper Address?
Gender bias is a pervasive issue that hinders social, economic, and political progress, particularly for women and marginalized groups. While social media offers new avenues for communication, it also amplifies gender bias. Existing methods for detecting gender bias often struggle with the nuanced, emotionally charged language found in social media and typically focus on identifying biased words or phrases without understanding the underlying causal connections. This gap makes it difficult to fully grasp the extent and mechanisms of gender bias, impeding efforts to combat it effectively and promote inclusive societies.
Methods
The authors propose a multi-task deep framework called GLIDE, which tackles both Gender-biased Language Identification (GLI) and Gender-biased Language Extraction (GLE). Key methodological contributions include:
- GLIdE Task: A novel task that goes beyond simple detection to identify the specific phrases or "causes" that contribute to gender bias.
- Multi-modal Fusion: The framework integrates several types of information:
- Textual Features: Utilizes BERT for rich contextual understanding of text.
- Emotional Features: Leverages pre-trained BERT to extract emotion class semantic features, capturing the emotional tone of the text.
- Gender-Stereotyped Knowledge Graph: A novel approach to incorporate external knowledge about gender stereotypes. This is achieved through structural embedding using TransE and a vocabulary graph constructed with GCN.
- Text2Graph (T2G) Attention: A cross-modality attention mechanism that allows the model to map textual features to the knowledge graph, capturing nuanced interactions between language and gender stereotypes.
- Zero-Shot Strategy: A zero-shot loss objective is employed to minimize the difference between text features and emotion label features, enabling effective model optimization without requiring explicit emotion labels for all training data.
Datasets
The paper introduces the Gender-biased Analysis Corpus (GAC), a new benchmark dataset comprising 12,432 social media posts. GAC is built by consolidating instances from three existing datasets: Workplace Sexism, Call Me Sexist, and EXIST@IberLEF. A key innovation of GAC is that it includes manually annotated spans for gender-biased content, a feature absent in previous datasets.
Key Insights and Ideas
- Causal Understanding is Crucial: Simply identifying biased words isn't enough. Understanding why something is biased (the causal phrases) is essential for effective intervention and analysis.
- External Knowledge Matters: Incorporating a knowledge graph of gender stereotypes significantly enhances the model's ability to detect and explain gender bias, especially in subtle or context-dependent cases.
- Emotion Adds Nuance: Emotional information, when integrated, helps the model better interpret the sentiment and intent behind potentially biased language.
- Social Media is a Rich, Complex Domain: The challenges of analyzing gender bias in social media highlight the need for sophisticated models that can handle informal language, emotional cues, and implicit biases.
Key Results
The proposed GLIDE framework achieved state-of-the-art results on the GAC dataset:
- GLI Task: Achieved an F1 score of 77.86%, outperforming the best baseline (CMSEKI) by 4.05% and BERT-HateXplain by 4.88%.
- GLE Task: Achieved a ROS score of 0.81, surpassing CMSEKI by 3 points and BERT-HateXplain by 5 points.
- Qualitative Analysis: The GLIDE model demonstrated a superior ability to identify gender-stereotyped causal phrases that other models, including advanced ones like BERT-HateXplain and CMSEKI, missed. For instance, it correctly identified "bitches can't drive" as a gender-stereotyped causal phrase, while others focused only on the explicit "sexist" label.
- Human Evaluation: The GLIDE framework scored higher in Knowledge Consistency, Informativeness, and Fluency compared to baselines, indicating that its extracted causal spans are more accurate, relevant, and well-formed.
Limitations and Future Work
- Limitations: The paper acknowledges that the current GAC dataset is primarily in English. The performance of the model might be affected by the specific characteristics of the social media platforms and the nuances of language used.
- Future Work: The authors suggest extending the GLIDE method to analyze gender bias in other languages and domains (e.g., online news, academic writing). They also propose expanding the GAC dataset to include more diverse data from various social media platforms and user demographics, as well as data from other domains, to provide a more comprehensive understanding of gender bias.
Summary updated