TL;DR
Researchers at Meta AI and UPC conducted a massive study of "added toxicity"—toxic content introduced by machine translation (MT) that wasn't in the source text—across 164 languages. By combining the HOLISTICBIAS dataset with interpretability tools, they discovered that added toxicity is primarily driven by two phenomena: model hallucinations and specific mistranslations, particularly in low-resource languages.
What Problem Does This Paper Address?
Standard machine translation evaluation metrics like BLEU or chrF measure overall quality but often fail to distinguish between "venial" errors (minor grammar slips) and "critical" errors that can cause real-world harm. One of the most damaging critical errors is added toxicity: when a neutral source sentence (e.g., "I love being chubby") is translated into something offensive or profane in the target language.
While previous work noted this phenomenon, this paper provides the first large-scale, deep dive into why it happens. The authors seek to move beyond just counting toxic outputs to understanding the underlying mechanics—whether the model is "hallucinating" toxic words out of thin air or simply picking the wrong, offensive word for a benign source term.
Methods
The study utilized the NLLB-200 model (a 3.3B parameter dense model) to translate the HOLISTICBIAS dataset from English into 164 target languages. To analyze the results, the team employed three key technical pillars:
- Toxicity Detection: They used wordlist-based detectors covering 200 languages, categorizing outputs as toxic if they contained entries from specific lists (profanity, insults, etc.).
- ALTI+ (Interpretability): This method measures "input attribution"—essentially, how much each source token vs. the previously generated target tokens contributes to a specific word in the translation. A low source contribution score is a strong proxy for hallucination.
- Robustness Analysis: They calculated Gini impurity across translations. By taking a sentence template and swapping out nouns (e.g., "I am a [parent/doctor/child]"), they measured how much the translation of a specific descriptor (like "disabled") changed. High impurity indicates an "unstable" translation, which the authors found correlates with toxicity.
Datasets
- HOLISTICBIAS: The primary evaluation set, consisting of over 472,000 English sentences. It uses templates to explore 13 demographic axes (e.g., ability, sexual orientation, religion).
- FLORES-200: A standard benchmark used here to measure baseline translation quality (chrF scores) and to show that added toxicity is much more prevalent in the "stress-test" environment of HOLISTICBIAS than in formal, standard text.
Key Insights and Ideas
- The Hallucination-Mistranslation Split: The authors found that added toxicity isn't a monolith. About 45.6% of toxic translations have a high source contribution, suggesting mistranslation (the model thinks a toxic word is the correct translation for a neutral one). The remainder often stems from hallucination, where the model ignores the source and generates toxic "noise."
- Low-Resource Vulnerability: Added toxicity is significantly higher in low-resource languages. When a model has less data to learn from, it is more prone to both unstable translations and hallucinations.
- Demographic Triggers: Certain axes are "magnets" for toxicity. Sexual orientation, gender/sex, and ability saw the highest rates of added toxicity. Interestingly, terms for parents and grandparents were among the most frequent "toxic nouns," likely due to how these terms appear in colloquial or offensive contexts in training data.
- Predicting Toxicity: The combination of low source contribution (hallucination) and high Gini impurity (instability) creates a "danger zone." Translations in this region are roughly twice as likely to be toxic as the baseline.
Key Results
- Prevalence: Across 164 languages, added toxicity rates varied from 0% to 5%. Most languages (all but 13) stayed below 1%.
- Language Leaders: The highest rates (>2%) were found in languages like Luo, Tswana, Yoruba, and Indonesian.
- Correlation: For 84% of languages, there was a statistically significant correlation between the level of source contribution and the likelihood of toxicity.
- Mitigation Success: The authors tested filtering training data to remove pairs where the target was toxic but the source was not. This "no_add" filter drastically reduced toxicity in the resulting models (e.g., reducing toxic counts in Basque from 1167 to 129) without significantly degrading overall translation quality (chrF).
Limitations and Future Work
The study acknowledges several hurdles:
- Wordlist Limitations: Word-based detection is "brute force" and lacks context. Human evaluation revealed high false-positive rates in languages like Simplified Chinese (59.2%) and Western Persian (35.8%), where benign words were flagged as toxic due to homographs or polysemy.
- English-Centricity: The study only translated from English. Toxicity patterns might differ significantly when translating between other language pairs.
- Template Constraints: HOLISTICBIAS is synthetic. While useful for control, it doesn't capture the full range of natural human conversation where toxicity might manifest differently.
Future research directions include exploring how the amount of toxicity in the original training data correlates with these errors and developing more context-aware toxicity classifiers for a wider range of languages.
Summary updated