Toxicity in Multilingual Machine Translation at Scale

Marta R. Costa-jussà, Eric Smith, Christophe Ropers, Daniel Licht, Jean Maillard, Javier Ferrando, Carlos Escolano

TL;DR

Researchers at Meta AI and UPC conducted a massive study of "added toxicity"—toxic content introduced by machine translation (MT) that wasn't in the source text—across 164 languages. By combining the HOLISTICBIAS dataset with interpretability tools, they discovered that added toxicity is primarily driven by two phenomena: model hallucinations and specific mistranslations, particularly in low-resource languages.

What Problem Does This Paper Address?

Standard machine translation evaluation metrics like BLEU or chrF measure overall quality but often fail to distinguish between "venial" errors (minor grammar slips) and "critical" errors that can cause real-world harm. One of the most damaging critical errors is added toxicity: when a neutral source sentence (e.g., "I love being chubby") is translated into something offensive or profane in the target language.

While previous work noted this phenomenon, this paper provides the first large-scale, deep dive into why it happens. The authors seek to move beyond just counting toxic outputs to understanding the underlying mechanics—whether the model is "hallucinating" toxic words out of thin air or simply picking the wrong, offensive word for a benign source term.

Methods

The study utilized the NLLB-200 model (a 3.3B parameter dense model) to translate the HOLISTICBIAS dataset from English into 164 target languages. To analyze the results, the team employed three key technical pillars:

  1. Toxicity Detection: They used wordlist-based detectors covering 200 languages, categorizing outputs as toxic if they contained entries from specific lists (profanity, insults, etc.).
  2. ALTI+ (Interpretability): This method measures "input attribution"—essentially, how much each source token vs. the previously generated target tokens contributes to a specific word in the translation. A low source contribution score is a strong proxy for hallucination.
  3. Robustness Analysis: They calculated Gini impurity across translations. By taking a sentence template and swapping out nouns (e.g., "I am a [parent/doctor/child]"), they measured how much the translation of a specific descriptor (like "disabled") changed. High impurity indicates an "unstable" translation, which the authors found correlates with toxicity.

Datasets

  • HOLISTICBIAS: The primary evaluation set, consisting of over 472,000 English sentences. It uses templates to explore 13 demographic axes (e.g., ability, sexual orientation, religion).
  • FLORES-200: A standard benchmark used here to measure baseline translation quality (chrF scores) and to show that added toxicity is much more prevalent in the "stress-test" environment of HOLISTICBIAS than in formal, standard text.

Key Insights and Ideas

  • The Hallucination-Mistranslation Split: The authors found that added toxicity isn't a monolith. About 45.6% of toxic translations have a high source contribution, suggesting mistranslation (the model thinks a toxic word is the correct translation for a neutral one). The remainder often stems from hallucination, where the model ignores the source and generates toxic "noise."
Figure 1: Examples of translations in English-to-French, English-to-Spanish or English-to-Catalan. Sentences show input attributions for bold words in the cases of hallucination (sentence 1); mistranslation (sentence 2); and a correct translation (sentence 3). We observe that the hallucination example focuses more in the target context than in the source sentence compared to the other two examples.
  • Low-Resource Vulnerability: Added toxicity is significantly higher in low-resource languages. When a model has less data to learn from, it is more prone to both unstable translations and hallucinations.
  • Demographic Triggers: Certain axes are "magnets" for toxicity. Sexual orientation, gender/sex, and ability saw the highest rates of added toxicity. Interestingly, terms for parents and grandparents were among the most frequent "toxic nouns," likely due to how these terms appear in colloquial or offensive contexts in training data.
  • Predicting Toxicity: The combination of low source contribution (hallucination) and high Gini impurity (instability) creates a "danger zone." Translations in this region are roughly twice as likely to be toxic as the baseline.
Figure 3: The toxicity of descriptors in translation varies greatly as a function of both the source contribution to and the robustness of the translation. Left: the population distribution of the translations across all languages and HOLISTICBIAS sentences. Right: the rate of toxicity of translations, with white representing no samples or 0% toxicity. A high Gini impurity indicates a low robustness in the translation of descriptors across different HOLISTICBIAS nouns. Several regions have high toxicity, but many of them have few samples. However, the region bounded by the cyan box has relatively high rates of toxicity as well as high numbers of samples.

Key Results

  • Prevalence: Across 164 languages, added toxicity rates varied from 0% to 5%. Most languages (all but 13) stayed below 1%.
  • Language Leaders: The highest rates (>2%) were found in languages like Luo, Tswana, Yoruba, and Indonesian.
  • Correlation: For 84% of languages, there was a statistically significant correlation between the level of source contribution and the likelihood of toxicity.
  • Mitigation Success: The authors tested filtering training data to remove pairs where the target was toxic but the source was not. This "no_add" filter drastically reduced toxicity in the resulting models (e.g., reducing toxic counts in Basque from 1167 to 129) without significantly degrading overall translation quality (chrF).

Limitations and Future Work

The study acknowledges several hurdles:

  • Wordlist Limitations: Word-based detection is "brute force" and lacks context. Human evaluation revealed high false-positive rates in languages like Simplified Chinese (59.2%) and Western Persian (35.8%), where benign words were flagged as toxic due to homographs or polysemy.
  • English-Centricity: The study only translated from English. Toxicity patterns might differ significantly when translating between other language pairs.
  • Template Constraints: HOLISTICBIAS is synthetic. While useful for control, it doesn't capture the full range of natural human conversation where toxicity might manifest differently.

Future research directions include exploring how the amount of toxicity in the original training data correlates with these errors and developing more context-aware toxicity classifiers for a wider range of languages.

Summary updated

Abstract

Machine Translation systems can produce different types of errors, some of which are characterized as critical or catastrophic due to the specific negative impact that they can have on users. In this paper we focus on one type of critical error: added toxicity. We evaluate and analyze added toxicity when translating a large evaluation dataset (HOLISTICBIAS, over 472k sentences, covering 13 demographic axes) from English into 164 languages. An automatic toxicity evaluation shows that added toxicity across languages varies from 0% to 5%. The output languages with the most added toxicity tend to be low-resource ones, and the demographic axes with the most added toxicity include sexual orientation, gender and sex, and ability. We also perform human evaluation on a subset of 8 translation directions, confirming the prevalence of true added toxicity. We use a measurement of the amount of source contribution to the translation, where a low source contribution implies hallucination, to interpret what causes toxicity. Making use of the input attributions allows us to explain toxicity, because the source contributions significantly correlate with toxicity for 84% of languages studied. Given our findings, our recommendations to reduce added toxicity are to curate training data to avoid mistranslations, mitigate hallucination and check unstable translations.
BibTeX
@inproceedings{costajussa2023toxicity,
  author = {Marta R. Costa-jussà and Eric Smith and Christophe Ropers and Daniel Licht and Jean Maillard and Javier Ferrando and Carlos Escolano},
  title = {{Toxicity in Multilingual Machine Translation at Scale}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=PT6lSdWEgw},
}
Figure