Abstract
Neural Machine Translation (NMT) has achieved high levels of fluency, yet it often struggles with faithfulness, leading to omissions or hallucinations. We present FENMT (Faithfulness Enhanced Neural Machine Translation), a novel training strategy designed to improve translation accuracy. FENMT identifies mistranslated fragments during training and utilizes a multi-task learning (MTL) framework to address encoding difficulties, contextual gaps, and language model dominance. Experimental results on WMT benchmarks show significant BLEU score improvements and a substantial reduction in critical translation errors, particularly in long-sentence scenarios.
Introduction
The primary goal of NMT is to produce translations that are both fluent and faithful to the source text. While modern architectures like the Transformer excel at fluency, they are prone to "unfaithfulness"—generating text that sounds natural but omits key source information or includes fabricated details. This paper identifies three root causes: the encoder's inability to represent complex source fragments, the decoder's failure to attend to relevant source context, and the tendency of the decoder's internal language model to override source-driven generation.
Methodology: The FENMT Framework
FENMT introduces a "closed-loop" feedback mechanism that allows the model to learn from its own mistakes during the training process.
Identifying Mistranslated Fragments
Unlike standard "teacher-forcing" training, FENMT incorporates a self-testing phase. At each epoch, the model translates a subset of the training data. These translations are compared against gold-standard references to identify "mistranslated fragments"—phrases that should have appeared in the output but were missed or incorrectly translated.
Multi-Task Learning (MTL) Paradigm
To correct these errors, the model is trained on three auxiliary tasks alongside the standard NMT objective:
- Encoder Masked Language Modeling (MLM): The system masks source words corresponding to the identified errors. The encoder must then predict these words, forcing it to develop more robust representations of "difficult" input segments.
- Cross-Attention Alignment: This task guides the decoder's attention mechanism to align more accurately with the source words that correspond to the target fragments, bridging contextual gaps.
- Max-Margin Objective: To counteract language model dominance, a contrastive learning approach is used. It penalizes the model for generating common, fluent words that are irrelevant to the source, while rewarding the generation of the correct, previously missed fragments.
Experimental Results
The researchers evaluated FENMT on three standard benchmarks: WMT17 Chinese-English (Zh→En), WMT14 English-German (En→De), and WMT16 English-Romanian (En→Ro).
Quantitative and Qualitative Gains
FENMT consistently outperformed the Transformer-Big baseline, achieving a gain of up to 1.44 BLEU points on the Zh→En task. Beyond automated metrics, human evaluation by professional translators revealed that FENMT reduced "Critical" translation errors by 50% and "Major" omissions by 23.8%. These results underscore the model's ability to produce more reliable translations in practice.
Performance on Long Sentences
One of the most significant findings is FENMT's stability when processing long sentences (30-60 words). Standard NMT models often suffer a drop in faithfulness as sentence length increases. In contrast, FENMT maintains high accuracy, as shown in the comparative analysis of BLEU scores across varying sentence lengths.
Discussion
A key insight from the ablation studies is the critical role of the encoder; the MLM task provided the most significant performance boost, suggesting that many faithfulness issues originate from poor initial representations. While the training process is slightly slower (approximately 0.86x to 0.94x the speed of standard training), the inference speed remains identical to a standard Transformer, making FENMT highly suitable for production environments.
Conclusion
FENMT offers a practical and effective strategy for enhancing the faithfulness of NMT systems. By actively identifying and addressing its own failures through a multi-task learning curriculum, the model achieves superior accuracy without sacrificing fluency or inference efficiency. Future research will explore the applicability of this "identify-and-fix" paradigm to other text generation tasks such as summarization and dialogue.
Summary updated