Introduction
Multilingual Neural Machine Translation (MNMT) models have revolutionized the field by enabling a single architecture to translate between numerous language pairs. These models are particularly valuable for zero-shot translation, where they translate between languages never paired during training. However, MNMT models frequently suffer from "off-target" errors, where the output is generated in the wrong language—often English or a copy of the source text.
The Multilingual Beam Search Curse
A counter-intuitive finding in MNMT research is that increasing the beam size during decoding—a standard technique to improve quality—actually degrades performance in zero-shot scenarios. This phenomenon, known as the "multilingual beam search curse," occurs because the model's probability scores for incorrect languages can eventually overtake the correct target language as the search space expands.
As shown in the data, increasing the beam size from 5 to 20 for German-to-French (De→Fr) translation causes the off-target rate to jump from 23.1% to 41.4%, while the BLEU score drops significantly. This suggests that the model's internal language steering is fragile and easily overwhelmed by high-probability paths in incorrect languages.
Language-informed Beam Search (LiBS)
To address this, we introduce Language-informed Beam Search (LiBS), a plug-and-play decoding algorithm. LiBS integrates an external Language Identification (LiD) model (specifically FastText) directly into the beam search process. The decoding objective is modified to include a language constraint:
$$Score = \log P_{NMT}(y|x) + \alpha \log P_{LiD}(Target\_Lang, y)$$
Here, $\alpha$ is a tunable coefficient that balances the NMT model's fluency with the LiD model's language accuracy. To maintain efficiency, LiBS employs a "pre-select window," applying the LiD scoring only to the top-$w$ candidates suggested by the NMT model (typically $w=2$).
The relationship between $\alpha$ and performance is critical. As $\alpha$ increases, the off-target rate decreases steadily, while the BLEU score improves as the model is successfully steered toward the correct language. This demonstrates that forcing the model into the correct language yields better semantic results than allowing it to drift into high-probability off-target languages like English.
Key Results and Insights
LiBS was evaluated on the WMT-10 and OPUS-100 benchmarks. On WMT-10, LiBS reduced the average off-target rate from 22.9% to 7.7%, yielding a +1.1 BLEU improvement. On the massive OPUS-100 dataset, the off-target rate dropped from 65.8% to 25.3%.
The success of LiBS stems from its ability to counteract "recency bias." MNMT models often forget the initial language constraint as a sentence grows. By providing a persistent language signal at each decoding step, LiBS ensures the model remains on-target. Crucially, LiBS "breaks" the beam search curse; with the language constraint in place, larger beam sizes no longer lead to catastrophic language drift.
Conclusion
Language-informed Beam Search provides a simple yet effective way to improve multilingual translation without retraining. By leveraging external language identification during inference, it significantly mitigates off-target errors, especially in zero-shot settings. While it introduces some computational overhead (roughly 3.5x to 7.5x slower than standard search), the gains in translation reliability make it a compelling choice for robust multilingual systems.
Summary updated