From Pink and Blue to a Rainbow Hue! Defying Gender Bias through Gender Neutralizing Text Transformations

Gopendra Vikram Singh, Soumitra Ghosh, Neil Dcruze, Asif EkbalIJCAI 2024

Introduction

Gender bias in textual data remains a pervasive challenge in Natural Language Processing (NLP). From subtle stereotypes to overt sexism, biased language in online environments perpetuates societal inequalities. While previous efforts have attempted to mitigate this through rule-based swaps (e.g., replacing "he" with "they"), these methods often fail to capture linguistic nuance, leading to unnatural phrasing or loss of original context.

To address these limitations, researchers have introduced GENETT (GEnder-NEutralizing Text Transformation), a novel deep-learning framework designed to transform gender-biased text into inclusive language while preserving the original intent. Alongside the model, the team released GENRE (GEnder Neutralized REvisions), the first large-scale dataset of human-annotated gender-neutral revisions, providing a robust benchmark for the field.

The GENETT Framework

GENETT is built upon a sophisticated architecture that treats gender neutralization as a style transfer task. The goal is to disentangle the "content" of a sentence from its "biased style" and re-infuse it with a "neutral style."

Overview of the GENETT framework including training phases, inference logic, and the Vector Quantization codebook.

Architectural Pillars

  1. BART-based Auto-encoders: The framework utilizes BART (Bidirectional and Auto-Regressive Transformers) to map biased and neutral text into a shared latent space. This ensures that the underlying semantic meaning is captured before any stylistic manipulation occurs.
  2. Vector Quantization (VQ): To effectively isolate stylistic elements, GENETT employs Vector Quantization. This process maps continuous text features into discrete "codebooks" ($F_{bias}$ and $F_{neutral}$). By discretizing the representation, the model can more precisely identify and manipulate the features associated with gender bias versus those associated with neutral language.
  3. Neutrality-Infused Stylization (NIS) Module: This is the core engine of the transformation. The NIS module takes the content from a biased sentence and "infuses" it with features extracted from the neutral codebook. During inference, a "neutrality slider" (controlled by parameters $\alpha$ and $\beta$) allows users to tune the balance between textual fidelity (staying close to the original wording) and neutralization (the degree of bias removal).

The NIS Module Mechanics

The NIS module is specifically designed to handle the complexities of style infusion. It utilizes an LSTM cell to process the biased input sequence, followed by a multi-head attention mechanism. This attention layer allows the model to selectively focus on relevant neutral features from the codebook, ensuring that the resulting transformation is both contextually relevant and stylistically neutral.

Detailed view of the Neutrality-Infused Stylization (NIS) module, highlighting the LSTM cell and Multi-Head Attention components.

The GENRE Dataset

A significant contribution of this work is the GENRE (GEnder Neutralized REvisions) corpus. This dataset consists of 5,230 instances of gender-biased text paired with high-quality, human-annotated neutral revisions.

To create GENRE, the researchers consolidated three existing benchmark sexist datasets: Workplace Sexism, Call Me Sexist, and EXIST@IberLEF. Linguists and computer scientists then manually rewrote these biased sentences. For example, a biased sentence like "Chicks just aren't that funny" was carefully revised to "Some people just aren't that funny," ensuring that the target of the statement was generalized without losing the original sentiment.

Key Insights and Results

GENETT was evaluated against ten state-of-the-art style transfer baselines using both automated metrics and human judgment. The results demonstrate a significant advancement in the field:

  • Content Preservation: GENETT achieved a BLEU score of 54.1, outperforming the next best model, DualRL (48.5), by a wide margin. This indicates that the model is exceptionally good at keeping the original meaning intact.
  • Fluency: With a Perplexity (PPL) of 1.24, the text generated by GENETT is highly natural and coherent, closely mimicking human speech patterns.
  • Human Evaluation: In blind tests, human evaluators rated GENETT higher than all competitors across three key dimensions: Fluency (3.36/5), Adequacy (3.41/5), and Informativeness (3.83/5).
  • Comparison with LLMs: While large language models like ChatGPT-3.5 are capable of neutralization, qualitative analysis showed that they often struggle to maintain the original tone or accidentally change the subject of the sentence. GENETT, being specialized for this task, maintains higher stability in the "target entity" of the sentence.

Limitations and Future Work

While GENETT represents a major step forward, the authors acknowledge several areas for improvement. Currently, the model primarily addresses binary gender bias due to the constraints of the source datasets. Future work aims to expand the framework to include non-binary gender identities and to adapt the model for languages other than English. Additionally, the researchers plan to explore the application of GENETT in more formal contexts, such as technical documentation and literature, beyond its current focus on cyber harassment and social media text.

Summary updated

BibTeX
@inproceedings{singh2024from,
  author = {Gopendra Vikram Singh and Soumitra Ghosh and Neil Dcruze and Asif Ekbal},
  title = {{From Pink and Blue to a Rainbow Hue! Defying Gender Bias through Gender Neutralizing Text Transformations}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure