Enhanced Zero-Shot Machine Translation via Fixed Prefix Pair Bootstrapping

Van-Hien Tran, Masao UtiyamaNAACL 2025

TL;DR

This paper introduces a simple yet effective method to improve zero-shot machine translation (MT) using Large Language Models (LLMs). By prepending a single, accurate bilingual word or phrase pair to both source and target sentences, the approach guides LLMs to generate more precise and on-target translations, significantly outperforming traditional zero-shot and one-shot baselines.

What Problem Does This Paper Address?

Large Language Models (LLMs) have shown remarkable capabilities in "in-context learning" (ICL), allowing them to perform tasks with just instructions and a few examples. However, in zero-shot scenarios (where no examples are provided), LLMs often struggle with calibration issues, leading to problems like hallucinations and "off-target" translations – where the output drifts into an unintended language. This is particularly problematic for machine translation, where accuracy and fidelity to the target language are paramount. While few-shot learning can improve performance, preparing examples is labor-intensive, especially for low-resource languages. This paper tackles the challenge of achieving reliable zero-shot MT without the need for extensive example preparation.

Methods

The core contribution of this paper is a novel prompting strategy called "Fixed Prefix Pair Bootstrapping." Instead of relying on full sentence examples or just instructions, the method introduces a single, accurate bilingual word or phrase pair (e.g., "As a result" / "Di conseguenza") as a prefix for both the source and target sentences.

Here's how it works:

  • Traditional Zero-Shot: The LLM receives only instructions (e.g., "Translate this English sentence to Italian").
  • One-Shot Baseline: The LLM receives instructions plus one full example sentence pair.
  • Proposed Approach: The LLM receives instructions, and the source sentence is prepended with a word/phrase from the source language (e.g., "As a result, I achieved a good outcome in the examination."), and the expected target output is also prefixed with its translation (e.g., "Di conseguenza, ...").

The key idea is that this fixed prefix acts as a strong signal, guiding the LLM to initiate the translation correctly and maintain consistency in the target language, thereby mitigating off-target errors. The chosen prefix is intended to be neutral and relevant, ensuring it doesn't disrupt the natural flow of the original sentence.

Datasets

The evaluation was conducted using the FLORES-101 devtest dataset. This dataset is a standard benchmark for low-resource and multilingual machine translation. The experiments specifically focused on four language directions: English → Italian, English → Vietnamese, English → Irish, and English → Portuguese.

Key Insights and Ideas

  • Simplicity is Powerful: The most striking insight is how a very simple addition – a single bilingual word/phrase prefix – can dramatically improve zero-shot MT performance. This suggests that LLMs are highly sensitive to initial contextual cues.
  • Targeting the Off-Target Problem: The method directly addresses the "off-target" issue, a known weakness in zero-shot LLM translation, by providing a clear anchor in the target language from the outset.
  • Low-Resource Advantage: The approach is particularly beneficial for low-resource languages, as it doesn't require extensive bilingual corpora or expert-curated example sets, which are often scarce for such languages.
  • Bootstrapping Translation Quality: The prefix acts as a "bootstrapping" mechanism, helping the LLM start the translation process on the right foot and maintain that momentum.

Key Results

The proposed "Fixed Prefix Pair Bootstrapping" method consistently outperformed both traditional zero-shot and one-shot baseline methods across all tested LLMs (Gemma-7B, LLaMA-2-7B, LLaMA-2-13B, LLaMA-3-8B) and language pairs.

  • Overall Improvement: The method achieved significant average gains, with improvements of up to 6.43 points in ChrF++ score for English-to-Irish.
  • Specific Gains: A notable result was a 15.59 point improvement for English-to-Irish using the LLaMA-2-13B model.
  • Robustness: The approach demonstrated stable improvements across all language pairs and models, indicating its robustness.
  • Qualitative Evidence: Table 2 shows a compelling case study where the baseline methods produced output in English, while the proposed method successfully generated translations in Italian.

Limitations and Future Work

  • Limited Scope: The current evaluation is limited to four language pairs and a general-domain dataset (FLORES-101).
  • Future Directions: The authors suggest extending the evaluation to a wider range of language pairs and domain-specific MT datasets (e.g., medical translation) to verify the broader applicability and effectiveness of their method.

Summary updated

BibTeX
@inproceedings{tran2025enhanced,
  author = {Van-Hien Tran and Masao Utiyama},
  title = {{Enhanced Zero-Shot Machine Translation via Fixed Prefix Pair Bootstrapping}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure