Multilingual Amnesia: On the Transferability of Unlearning in Multilingual LLMs

Alireza dehghanpour farashah, Aditi Khandelwal, Marylou Fauchard, Zhuan Shi, Negar Rostamzadeh, Golnoosh Farnadi

Abstract

As multilingual large language models (LLMs) become more widely used, ensuring their safety and fairness across diverse linguistic contexts presents unique challenges. While existing research on machine unlearning has primarily focused on monolingual settings, typically English, multilingual environments introduce additional complexities due to cross-lingual knowledge transfer and biases embedded in both pretraining and fine-tuning data. This work investigates multilingual unlearning using the Aya-Expanse 8B model under two settings: (1) data unlearning and (2) concept unlearning. We extend benchmarks for factual knowledge and stereotypes to ten languages: English, French, Arabic, Japanese, Russian, Farsi, Korean, Hindi, Hebrew, and Indonesian. Our experiments demonstrate that unlearning in high-resource languages is generally more stable, with asymmetric transfer effects observed between typologically related languages. Furthermore, syntactic similarity is identified as the strongest predictor of cross-lingual unlearning behavior.

Introduction

As Large Language Models (LLMs) become central to global digital infrastructure, they must comply with safety requirements and regulations such as the "right to be forgotten" (e.g., GDPR). Machine unlearning aims to remove specific facts or harmful biases from a model's weights after training. However, most unlearning research is conducted in English, leaving a significant gap in our understanding of how these interventions affect multilingual models.

Multilingual LLMs share a single parameter space to encode many languages. A critical question arises: If a model is taught to forget a fact in English, does it still "remember" that information in French, Hindi, or Arabic? This phenomenon, which we term "multilingual amnesia," is often incomplete. As shown in Figure 1, a model may successfully refuse a harmful prompt in English and French after unlearning, yet still provide a prohibited response when queried in Hindi.

Conceptual illustration of unlearning failure across languages.

Methodology

We evaluated three gradient-based unlearning algorithms on the Aya-Expanse 8B model:

  1. Gradient Difference (GradDiff): Minimizes the likelihood of generating correct answers for the "forget set" while maximizing accuracy for the "retain set."
  2. GradDiff with KL (GradDiff-KL): Incorporates a Kullback–Leibler (KL) divergence term to maintain the model's general reasoning capabilities and prevent performance collapse.
  3. Negative Preference Optimization (NPO): Frames unlearning as a preference alignment task, treating the "forget" data as a highly undesirable response.

Datasets and Tasks

We extended two major benchmarks into ten languages spanning five language families: English (en), French (fr), Arabic (ar), Japanese (ja), Russian (ru), Farsi (fa), Korean (ko), Hindi (hi), Hebrew (iw), and Indonesian (id).

  • Data Unlearning (TOFU): Testing the surgical removal of synthetic personal data (author profiles).
  • Concept Unlearning (SeeGULL): Mitigating broad geocultural stereotypes, reformulated into a multilingual Multiple Choice Question (MCQ) format.

Results and Analysis

Our experiments reveal that unlearning is predominantly language-specific. Knowledge often remains tied to the linguistic context in which it was originally learned, suggesting that multilingual models do not store all concepts in a purely language-agnostic manner.

Methodological Comparison

Among the tested methods, NPO emerged as the most robust. It effectively targeted the forget set while maintaining high utility across the retain set. Figure 3 illustrates the cross-lingual transfer patterns for all three methods. While transfer is generally limited, certain clusters of languages show higher degrees of shared "forgetting."

Comparison of cross-lingual unlearning transfer across GradDiff, GradDiff-KL, and NPO methods.

Predictors of Transfer

We analyzed various linguistic distances to determine what drives cross-lingual transfer. Syntactic similarity (grammar and sentence structure) proved to be the strongest predictor, with significant correlations ($r = 0.347$ to $0.399$) across all methods. This outperformed phonological or purely lexical similarities.

Asymmetry and Stability

We observed "Asymmetric Amnesia," where transfer is not always reciprocal. For instance, unlearning in lower-resource languages like Indonesian sometimes impacted the English version more strongly than vice versa. Furthermore, unlearning in high-resource languages (English, French) was significantly more stable. Interventions in lower-resource languages (e.g., Farsi) were more likely to cause "catastrophic forgetting," where the model's general performance on unrelated tasks degraded sharply.

Conclusion

This study highlights that unlearning in one language is not a universal safety patch for multilingual LLMs. While broad "concept" unlearning (like stereotype mitigation) transfers more readily than specific "data" unlearning, the effects are largely governed by syntactic proximity and language resource levels. Future work should focus on "language-aware" unlearning strategies that explicitly account for cross-lingual interactions, potentially utilizing "pivot" languages to ensure consistent safety across the model's entire linguistic repertoire.

Summary updated

Abstract

As multilingual large language models become more widely used, ensuring their safety and fairness across diverse linguistic contexts presents unique challenges. While existing research on machine unlearning has primarily focused on monolingual settings, typically English, multilingual environments introduce additional complexities due to cross-lingual knowledge transfer and biases embedded in both pretraining and fine-tuning data. In this work, we study multilingual unlearning using the Aya-Expanse 8B model under two settings: (1) data unlearning and (2) concept unlearning. We extend benchmarks for factual knowledge and stereotypes to ten languages through translation: English, French, Arabic, Japanese, Russian, Farsi, Korean, Hindi, Hebrew, and Indonesian. These languages span five language families and a wide range of resource levels. Our experiments show that unlearning in high-resource languages is generally more stable, with asymmetric transfer effects observed between typologically related languages. Furthermore, our analysis of linguistic distances indicates that syntactic similarity is the strongest predictor of cross-lingual unlearning behavior.
BibTeX
@inproceedings{farashah2026multilingual,
  author = {Alireza dehghanpour farashah and Aditi Khandelwal and Marylou Fauchard and Zhuan Shi and Negar Rostamzadeh and Golnoosh Farnadi},
  title = {{Multilingual Amnesia: On the Transferability of Unlearning in Multilingual LLMs}},
  booktitle = {EACL (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.260},
}
Figure