Easy Adaptation to Mitigate Gender Bias in Multilingual Text Classification

Xiaolei HuangNAACL 2022 · DOI

Abstract

Machine learning models often amplify human biases present in training data, leading to discriminatory outcomes. While fairness-aware NLP has gained traction, research remains predominantly focused on English. This paper introduces a simple yet effective framework to reduce gender bias in multilingual text classifiers by treating gender as a domain adaptation problem. By applying the "Frustratingly Easy Domain Adaptation" (FEDA) framework, the researchers achieved significant fairness improvements—reducing bias by up to 57%—across eight different languages in hate speech detection and sentiment analysis tasks, often without sacrificing predictive accuracy.

Introduction

The challenge of multilingual bias mitigation lies in the diverse ways bias manifests across languages, influenced by variations in grammar (e.g., gendered nouns) and cultural context. Existing debiasing techniques are often complex or data-intensive. This research addresses the need for a standard, accessible method to ensure classifiers perform equitably for different gender groups across various linguistic settings.

Methodology

The authors propose using the Easy Adaptation Framework, based on "Frustratingly Easy Domain Adaptation" (FEDA). This approach treats gender (e.g., male vs. female) as distinct domains.

Instead of feeding a single text feature to the model, the feature is tripled into three components:

  1. General version: Shared across all genders.
  2. Male-specific version: Active only if the author is identified as male.
  3. Female-specific version: Active only if the author is identified as female.

During the training step, the model learns to distinguish between universal features and those specific to a gender group. During the testing step, the model is restricted to using only the general features for prediction, effectively filtering out gender-specific signals that lead to biased classifications.

The Easy Adaptation Framework (FEDA) illustrating the training and testing steps for gender bias mitigation.

This framework was evaluated across three model architectures: Logistic Regression (LR), Recurrent Neural Networks (RNN/GRU), and multilingual BERT.

Experimental Setup

The study utilized two public multilingual datasets with gender annotations:

  • Hate Speech (HS): Twitter data covering English (EN), Italian (IT), Portuguese (PT), and Spanish (ES).
  • Online Reviews: Trustpilot reviews covering English (EN), French (FR), German (DE), and Danish (DA).

Performance was compared against three "fairness-aware" baselines: Blind (masking gendered words), Instance Weighting (adjusting sample importance), and Adversarial Training (training the model to ignore gender).

Results and Key Insights

Fairness was measured using Equality Differences (ED), where a lower score indicates a fairer model. The proposed adaptation method (Ours-DA) consistently improved group fairness by 14% to 57.7% over standard baselines.

Detailed performance metrics (F1-macro, AUC, and Fair) across multiple languages and datasets comparing the proposed method against various baselines.

Key findings include:

  • Fairness without Sacrifice: In many cases, the adaptation method maintained or even slightly improved macro F1 scores while significantly reducing bias. For example, on the Portuguese Hate Speech dataset, BERT-DA reduced the "Fair" score from 19.9 to 9.0 while increasing the F1-macro score.
  • Model Complexity: While BERT-based models provided the highest overall performance, RNN-based models often achieved better absolute fairness scores, suggesting that simpler architectures might be more amenable to certain debiasing techniques.
  • Cross-Lingual Applicability: The method proved effective across all eight languages, establishing its robustness as a benchmark for multilingual group fairness.

Conclusion and Future Work

This work demonstrates that framing bias mitigation as a domain adaptation problem is a powerful and accessible strategy for multilingual NLP. However, several limitations remain. The current study is constrained to binary gender due to dataset limitations; future research must incorporate non-binary identities. Additionally, exploring intersectionality—how this method handles multiple demographic factors like gender and race simultaneously—remains a critical next step for developing truly equitable global AI systems.

Summary updated

BibTeX
@inproceedings{huang2022easy,
  author = {Xiaolei Huang},
  title = {{Easy Adaptation to Mitigate Gender Bias in Multilingual Text Classification}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.naacl-main.52},
}
Figure