Abstract
SimNMT is a novel framework designed to translate a single source language into multiple target languages simultaneously by enabling real-time interaction between decoders. By leveraging both historical and "future" information through bidirectional decoding across different languages, SimNMT achieves significantly higher translation quality and more balanced sentence structures compared to traditional independent decoders.
Introduction
In Multilingual Neural Machine Translation (MNMT), the objective is often to translate a source language (e.g., English) into multiple targets (e.g., French and German) concurrently. Traditional systems typically treat these target languages as independent processes, failing to exploit two key opportunities:
- Inter-lingual Synergy: Semantic clues in one target language can often clarify ambiguities in another.
- The Suffix Problem: Standard left-to-right (L2R) decoding often results in higher accuracy at the beginning of a sentence (prefix) than at the end (suffix), leading to unbalanced outputs.
SimNMT addresses these issues by creating a decoder that integrates information from all target languages and both decoding directions (forward and backward) at every step.
Methodology: The SimNMT Framework
The core of SimNMT lies in its ability to synchronize and share information across decoding threads.
Synchronous Interactive Multilingual Attention (SimAtt)
SimAtt replaces the standard self-attention layer in the Transformer decoder. It allows the model to attend to four distinct information streams for any given target language:
- Intra-lingual History & Future: Tokens already generated (L2R) and tokens being generated by a parallel backward decoder (R2L) for the same language.
- Inter-lingual History & Future: Tokens generated in the other target language (L2R) and its corresponding backward decoder (R2L).
Model Architecture
The overall architecture incorporates these interactive attention blocks within a standard Transformer-based encoder-decoder structure, modified to handle multiple outputs simultaneously.
SIM Beam Search and Data Augmentation
A specialized SIM Beam Search algorithm maintains both forward and backward hypotheses for all target languages, ensuring synchronization. To train the model effectively despite the scarcity of trilingual parallel data, the authors utilized back-translation and pseudo-labeling to create synthetic trilingual corpora.
Experiments and Results
The model was evaluated on the IWSLT datasets (TED talks) for English to German/French and English to Chinese/Japanese.
Performance Gains
SimNMT achieved an average BLEU score improvement of +2.32 over the standard Transformer and +0.60 over the competitive GNMT-Multi baseline. Notably, the model showed a significant boost in "Suffix Accuracy" (the last 4 tokens of a sentence), confirming that bidirectional interaction effectively mitigates the unbalanced output problem common in L2R decoders.
Insights on Fusion
The researchers found that a ReLU-based non-linear fusion strategy for the four attention streams outperformed simple addition. This suggests that the model successfully learns to dynamically weight the most "trustworthy" information source at each decoding step.
Conclusion
SimNMT demonstrates that simultaneous, interactive decoding across languages and directions can significantly enhance translation quality. By treating multiple target languages as mutual constraints rather than independent tasks, the framework provides a more robust and balanced translation output. Future work will explore the scalability of this approach to a larger number of target languages and more sophisticated fusion mechanisms.
Summary updated