UNLEARNING GEO-CULTURAL STEREOTYPES IN MULTILINGUAL LLMS

Alireza Dehghanpour Farashah, Aditi Khandelwal, Negar Rostamzadeh, Golnoosh FarnadiBuildingTrust @ ICLR 2025 · OpenReview

TL;DR

This paper investigates how removing cultural stereotypes from Large Language Models (LLMs) in one language affects their behavior in other languages. By adapting the SeeGULL dataset for English, French, and Hindi, they show that unlearning biases in English can transfer to other languages, but the effectiveness depends on the LLM's multilingual training strategy. This work highlights a critical gap in English-centric LLM safety evaluations and proposes a method for improving global fairness.

What Problem Does This Paper Address?

The rapid adoption of multilingual LLMs has outpaced our ability to evaluate and mitigate their biases across diverse linguistic and cultural contexts. Current safety and fairness evaluations predominantly focus on English, leaving a significant blind spot for geoculturally situated stereotypes that can hinder global inclusivity. These biases, embedded in training data, can lead LLMs to perpetuate harmful stereotypes related to nationality, culture, and other identity markers. While methods like counterfactual data augmentation and unlearning exist, a comprehensive cross-cultural evaluation framework is lacking, making it difficult to assess their true effectiveness in multilingual settings. This paper aims to address this by exploring the cross-lingual transfer of stereotype unlearning.

Methods

The core of this work involves adapting and applying a machine unlearning technique to mitigate geocultural stereotypes in multilingual LLMs.

  • Stereotype Unlearning: The authors employ a gradient ascent (GA) approach, inspired by prior work (Yao et al., 2024), to "unlearn" undesirable stereotypical responses. This method uses negative samples to guide the model away from generating harmful content.
  • Objective Function: The unlearning process is guided by a multi-objective loss function:
  • L_fgt: Cross-entropy loss on stereotypical answers from the SeeGULL dataset to encourage unlearning.
  • L_retain: Cross-entropy loss on neutral responses to promote unbiased answers.
  • L_nor: KL divergence regularization term to keep the updated model close to the pretrained model, preventing degradation of overall performance on neutral queries. (Note: This term was dropped for unlearning in Hindi and French due to the lack of a corresponding English-only dataset like TruthfulQA).
  • Cross-Lingual Evaluation: The key innovation is evaluating the impact of unlearning in one language (e.g., English) on the model's responses in other languages (e.g., French and Hindi).

Datasets

  • SeeGULL Dataset: The primary dataset used is SeeGULL (Jha et al., 2023; Bhutani et al., 2024), which contains over 7,000 stereotypes from 178 countries across 8 geopolitical regions.
  • Adapted SeeGULL Dataset: The authors adapted SeeGULL into a multilingual question-answering (QA) format. This involved:
  • Pairing stereotypical attributes with associated identities.
  • Creating multiple-choice questions with plausible distractors and an "Unknown" option.
  • Translating the question portions into French and Hindi using Google Translate, followed by human verification. The answer options remained consistent across languages.
  • TruthfulQA Dataset: Used for the KL divergence regularization term (L_nor) to maintain model performance on neutral queries.

Key Insights and Ideas

  • Cross-Lingual Transfer is Real, But Uneven: Unlearning stereotypes in English does show a positive impact on reducing biased responses in French and Hindi. This suggests that multilingual LLMs are not entirely isolated in their linguistic processing.
  • Model Architecture and Training Matter: The Aya-Expanse-8B model, trained with a more diverse multilingual strategy (including synthetic and machine-translated data), exhibited significantly stronger cross-lingual transfer of unlearning compared to the Llama-3.1-8B model. This highlights that the way a model is trained multilingually is crucial for effective bias mitigation across languages.
  • Unlearning in Non-English Languages is Tricky: For the Llama model, unlearning in Hindi resulted in random, meaningless responses. This suggests that the gradient ascent method might be too coarse for languages with less training data or that the specific unlearning process needs adaptation for different linguistic contexts.
  • Linguistic Similarity and Data Volume Play a Role: The paper speculates that the greater linguistic similarity between English and French, along with potentially larger French training data, contributes to more pronounced unlearning effects in French compared to Hindi.

Key Results

  • English Unlearning Impact: For the Llama model, unlearning on English reduced stereotypical responses and increased "Unknown" selections in English, French, and Hindi, with similar but less pronounced improvements in French and Hindi.
  • Aya Model's Stronger Transfer: The Aya model showed a dramatic drop in stereotypical responses after unlearning on English: from 16% to 3% in English, 13% to 6% in French, and 13% to 7% in Hindi. Unbiased responses increased from 32% to 63%.
  • Impact of Unlearning in French/Hindi: Unlearning in French with Aya reduced stereotypes in English and Hindi. Unlearning in Hindi also reduced biases in English and French, though the overall effect was less pronounced than unlearning in English or French.
  • Performance on Neutral Tasks: Table 1 shows that the unlearning process generally maintained the models' performance on standard NLP benchmarks (MRPC, QQP, RTE, SST2), with only minor fluctuations, indicating that the unlearning did not significantly degrade overall capabilities.

Limitations and Future Work

  • Limited Language Coverage: The study is limited to English, French, and Hindi. Future work should extend this to more languages, potentially categorizing them by linguistic similarity and data volume to understand transfer dynamics better.
  • Unlearning Challenges in Low-Resource Languages: The difficulty encountered with unlearning in Hindi for the Llama model suggests that current methods may not be directly applicable to all languages without adaptation.
  • Model-Specific Transfer: The significant difference in cross-lingual transfer between Llama and Aya highlights the need to understand how different multilingual training strategies impact unlearning.
  • Future Directions: The authors suggest exploring reinforcement-based debiasing or meta-learning approaches as complementary solutions for enhancing fairness in multilingual AI systems. They also propose a systematic investigation into linguistic similarities and data volumes to guide future strategy development.

Summary updated

Abstract

As multilingual generative models become more widely used, most safety and fairness evaluation techniques still focus on English-language resources, while overlooking important cross-cultural factors. This limitation raises concerns about fairness and safety, particularly regarding geoculturally situated stereotypes that hinder the models’ global inclusivity. In this work, we present preliminary findings on the impact of stereotype unlearning across languages, specifically in English, French, and Hindi. Using an adapted version of the SeeGULL dataset, we analyze how unlearning stereotypes in one language influences other languages within multilingual large language models. Our study evaluates two model families, Llama-3.1-8B and Aya-Expanse-8B, to assess whether unlearning in one linguistic context transfers across languages, potentially mitigating or exacerbating biases in multilingual settings.
BibTeX
@inproceedings{farashah2025unlearning,
  author = {Alireza Dehghanpour Farashah and Aditi Khandelwal and Negar Rostamzadeh and Golnoosh Farnadi},
  title = {{UNLEARNING GEO-CULTURAL STEREOTYPES IN MULTILINGUAL LLMS}},
  booktitle = {BuildingTrust @ ICLR},
  year = {2025},
  url = {https://openreview.net/forum?id=xo2e14bczv},
}
Figure