Language-Informed Beam Search Decoding for Multilingual Machine Translation

Yilin Yang, Stefan Lee, Prasad Tadepalli

Introduction

Multilingual Neural Machine Translation (MNMT) models have revolutionized the field by enabling a single architecture to translate between numerous language pairs. These models are particularly valuable for zero-shot translation, where they translate between languages never paired during training. However, MNMT models frequently suffer from "off-target" errors, where the output is generated in the wrong language—often English or a copy of the source text.

The Multilingual Beam Search Curse

A counter-intuitive finding in MNMT research is that increasing the beam size during decoding—a standard technique to improve quality—actually degrades performance in zero-shot scenarios. This phenomenon, known as the "multilingual beam search curse," occurs because the model's probability scores for incorrect languages can eventually overtake the correct target language as the search space expands.

The multilingual beam search curse: increasing beam size leads to higher off-target rates and lower BLEU scores.

As shown in the data, increasing the beam size from 5 to 20 for German-to-French (De→Fr) translation causes the off-target rate to jump from 23.1% to 41.4%, while the BLEU score drops significantly. This suggests that the model's internal language steering is fragile and easily overwhelmed by high-probability paths in incorrect languages.

Language-informed Beam Search (LiBS)

To address this, we introduce Language-informed Beam Search (LiBS), a plug-and-play decoding algorithm. LiBS integrates an external Language Identification (LiD) model (specifically FastText) directly into the beam search process. The decoding objective is modified to include a language constraint:

$$Score = \log P_{NMT}(y|x) + \alpha \log P_{LiD}(Target\_Lang, y)$$

Here, $\alpha$ is a tunable coefficient that balances the NMT model's fluency with the LiD model's language accuracy. To maintain efficiency, LiBS employs a "pre-select window," applying the LiD scoring only to the top-$w$ candidates suggested by the NMT model (typically $w=2$).

Impact of the linear coefficient $\alpha$ on BLEU score and off-target rate.

The relationship between $\alpha$ and performance is critical. As $\alpha$ increases, the off-target rate decreases steadily, while the BLEU score improves as the model is successfully steered toward the correct language. This demonstrates that forcing the model into the correct language yields better semantic results than allowing it to drift into high-probability off-target languages like English.

Key Results and Insights

LiBS was evaluated on the WMT-10 and OPUS-100 benchmarks. On WMT-10, LiBS reduced the average off-target rate from 22.9% to 7.7%, yielding a +1.1 BLEU improvement. On the massive OPUS-100 dataset, the off-target rate dropped from 65.8% to 25.3%.

The success of LiBS stems from its ability to counteract "recency bias." MNMT models often forget the initial language constraint as a sentence grows. By providing a persistent language signal at each decoding step, LiBS ensures the model remains on-target. Crucially, LiBS "breaks" the beam search curse; with the language constraint in place, larger beam sizes no longer lead to catastrophic language drift.

Conclusion

Language-informed Beam Search provides a simple yet effective way to improve multilingual translation without retraining. By leveraging external language identification during inference, it significantly mitigates off-target errors, especially in zero-shot settings. While it introduces some computational overhead (roughly 3.5x to 7.5x slower than standard search), the gains in translation reliability make it a compelling choice for robust multilingual systems.

Summary updated

Abstract

Beam search decoding is the de-facto method for decoding auto-regressive Neural Machine Translation (NMT) models, including multilingual NMT where the target language is specified as an input. However, decoding multilingual NMT models commonly produces ``off-target'' translations -- yielding translation outputs not in the intended language. In this paper, we first conduct an error analysis of off-target translations for a strong multilingual NMT model and identify how these decodings are produced during beam search. We then propose Language-informed Beam Search (LiBS), a general decoding algorithm incorporating an off-the-shelf Language Identification (LiD) model into beam search decoding to reduce off-target translations. LiBS is an inference-time procedure that is NMT-model agnostic and does not require any additional parallel data. Results show that our proposed LiBS algorithm on average improves +1.1 BLEU and +0.9 BLEU on WMT and OPUS datasets, and reduces off-target rates from 22.9\% to 7.7\% and 65.8\% to 25.3\% respectively.
BibTeX
@inproceedings{yang2024language,
  author = {Yilin Yang and Stefan Lee and Prasad Tadepalli},
  title = {{Language-Informed Beam Search Decoding for Multilingual Machine Translation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.932},
}
Figure