Towards Multilingual Transitivity and Bidirectional Multilingual Agreement for Multilingual Document-level Machine Translation

Hongyuan Lu, Haoyang Huang, Shuming Ma, Dongdong Zhang, Furu Wei, Wai Lam

Abstract

Despite the fact that multilingual agreement (MA) has shown its importance for multilingual neural machine translation (MNMT), current methodologies in the field have two shortages: (i) they require parallel data between multiple language pairs, which is not always realistic, and (ii) they optimize the agreement in an ambiguous direction, which hampers translation performance. We present Bidirectional Multilingual Agreement via Switched Back-translation (BMA-SBT), a novel and universal multilingual agreement framework for fine-tuning pre-trained MNMT models. BMA-SBT (i) exempts the need for aforementioned parallel data by using a novel method called switched BT that creates synthetic text written in another source language using the translation target and (ii) optimizes the agreement bidirectionally with the Kullback-Leibler Divergence loss. Experiments indicate that BMA-SBT clearly improves the strong baselines on the task of MNMT with three benchmarks: TED Talks, News, and Europarl. In-depth analyses indicate that BMA-SBT brings additive improvements to the conventional BT method.

Introduction: The Challenge of Multilingual Agreement

In Multilingual Neural Machine Translation (MNMT), a single model is tasked with translating across multiple language pairs. A core objective in this field is achieving Multilingual Agreement (MA). The intuition is straightforward: if two sentences in different source languages (e.g., Chinese and German) convey the same meaning, their translations into a common target language (e.g., English) should be identical.

Conceptual illustration of Multilingual Agreement where different source languages provide complementary linguistic cues.

However, implementing effective MA has historically faced two significant barriers. First is Data Scarcity. Traditional MA methods require "multi-parallel" or "triangulated" data—datasets where the same content is available in three or more languages simultaneously. Such data is rare and expensive to produce. Second is Ambiguous Directionality. Previous approaches often relied on word-level code-switching or unidirectional alignment, which can be mathematically "blurry" and fail to fully exploit the unique linguistic strengths of each source language, such as tense markers in German or aspect markers in Chinese.

Methodology: BMA-SBT

To overcome these limitations, the authors propose Bidirectional Multilingual Agreement via Switched Back-translation (BMA-SBT). This framework is designed for fine-tuning large-scale pre-trained MNMT models like M2M-100 or mBART.

The BMA-SBT architecture featuring Switched Back-translation and Bidirectional KL Loss.

Switched Back-translation (SBT)

The first innovation, SBT, eliminates the need for authentic multi-parallel data. Instead of searching for rare datasets, the model generates its own synthetic auxiliary source. Given a target sentence y (e.g., in English) from a standard parallel pair (x, y), the model back-translates y into an auxiliary language z (e.g., German). This creates a synthetic triplet (x, z, y) where x and z are both sources for the same target y.

Bidirectional Multilingual Agreement (BMA)

The second innovation is the use of Kullback-Leibler (KL) Divergence to align the output probability distributions of the original source x and the synthetic auxiliary source z. Unlike previous methods, this alignment is bidirectional. This ensures that the translation path from the original source learns from the auxiliary source and vice versa. For instance, if the original source is Chinese (which lacks explicit tense) and the auxiliary is German (which has explicit tense), the bidirectional loss forces the Chinese-to-English translation to "attend" to the temporal cues present in the German-to-English path.

Experimental Setup

The BMA-SBT framework was evaluated on three major benchmarks for "Many-to-English" (X → En) translation:

  1. TED Talks (IWSLT15): A diverse dataset used for both sentence-level and document-level evaluation across six languages (French, German, Chinese, Vietnamese, Czech, and Thai).
  2. News Benchmark (News-commentary v11): Focused on French, German, Chinese, and Czech.
  3. Europarl (v7): A large-scale corpus of European Parliament proceedings covering nine European languages.

Key Results and Analysis

BMA-SBT demonstrated superior performance across all benchmarks and language pairs. On the TED Talks benchmark, BMA-SBT achieved an average BLEU score of 40.85, significantly outperforming the Baseline+BT (39.65) and the conventional Multilingual Agreement approach (40.27).

Ablation studies confirmed that removing either direction of the KL loss resulted in a measurable drop in performance, proving that the bidirectional nature of the agreement is essential for maximizing the transfer of linguistic information between languages. Furthermore, case studies revealed that BMA-SBT is particularly effective at resolving ambiguities related to noun plurality and verb tense by leveraging the context provided by the auxiliary language.

Conclusion

BMA-SBT presents a universal and effective framework for enhancing multilingual machine translation. By leveraging synthetic data through Switched Back-translation and enforcing consistency through Bidirectional KL Loss, it allows models to overcome data scarcity and exploit cross-lingual synergies. This approach proves that even without massive multi-parallel corpora, we can significantly improve translation quality by forcing models to reach a "multilingual consensus."

Summary updated

Abstract

Despite the fact that multilingual agreement (MA) has shown its importance for multilingual neural machine translation (MNMT), current methodologies in the field have two shortages: (i) require parallel data between multiple language pairs, which is not always realistic and (ii) optimize the agreement in an ambiguous direction, which hampers the translation performance. We present \textbf{B}idirectional \textbf{M}ultilingual \textbf{A}greement via \textbf{S}witched \textbf{B}ack-\textbf{t}ranslation (\textbf{BMA-SBT}), a novel and universal multilingual agreement framework for fine-tuning pre-trained MNMT models, which (i) exempts the need for aforementioned parallel data by using a novel method called switched BT that creates synthetic text written in another source language using the translation target and (ii) optimizes the agreement bidirectionally with the Kullback-Leibler Divergence loss. Experiments indicate that BMA-SBT clearly improves the strong baselines on the task of MNMT with three benchmarks: TED Talks, News, and Europarl. In-depth analyzes indicate that BMA-SBT brings additive improvements to the conventional BT method.
BibTeX
@misc{lu2022towards,
  author = {Hongyuan Lu and Haoyang Huang and Shuming Ma and Dongdong Zhang and Furu Wei and Wai Lam},
  title = {{Towards Multilingual Transitivity and Bidirectional Multilingual Agreement for Multilingual Document-level Machine Translation}},
  year = {2022},
  eprint = {2209.13940},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.13940},
}
Figure