Abstract
Sequence-to-sequence models are commonly trained via maximum likelihood estimation (MLE). However, standard MLE training considers a word-level objective, predicting the next word given the previous ground-truth partial sentence. This procedure focuses on modeling local syntactic patterns, and may fail to capture long-range semantic structure. We present a novel solution to alleviate these issues. Our approach imposes global sequence-level guidance via new supervision based on optimal transport, enabling the overall characterization and preservation of semantic features. We further show that this method can be understood as a Wasserstein gradient flow trying to match our model to the ground truth sequence distribution. Extensive experiments are conducted to validate the utility of the proposed approach, showing consistent improvements over a wide variety of NLP tasks, including machine translation, abstractive text summarization, and image captioning.
Introduction
Sequence-to-sequence (Seq2Seq) models have revolutionized natural language processing, powering applications from machine translation to text summarization. Despite their success, the standard training paradigm—Maximum Likelihood Estimation (MLE)—suffers from two significant drawbacks: exposure bias and metric mismatch. Exposure bias arises because models are trained on ground-truth prefixes but must generate sequences from their own potentially erroneous predictions during inference. Metric mismatch occurs because MLE optimizes for local, position-specific word accuracy, whereas evaluation metrics like BLEU or ROUGE prioritize global semantic coherence.
To address these challenges, we propose a novel training objective that incorporates Optimal Transport (OT). By treating sentences as distributions of semantic embeddings, OT provides a global, sequence-level supervision signal that complements the local syntactic focus of MLE.
Methodology
The proposed framework augments the standard MLE loss with an OT-based loss ($\mathcal{L}_{ot}$). Unlike MLE, which requires exact word-for-word alignment, OT calculates the minimum "cost" to transform the model's generated sequence distribution into the target sequence distribution in an embedding space.
Optimal Transport Framework
The core of our approach is the computation of the Wasserstein distance between the generated and target sequences. To ensure the objective is differentiable and computationally efficient, we employ the Inexact Proximal point method for Optimal Transport (IPOT). This algorithm provides a stable and efficient way to compute the transport plan $T$ and the cost matrix $C$.
Technical Innovations
- Soft-argmax: To bridge the gap between discrete word selection and differentiable optimization, we use a soft-argmax operator to represent the model's belief over the vocabulary.
- Soft-copying Mechanism: We introduce an additional OT loss between the generated output and the source input. This acts as a semantic "copy" mechanism, ensuring the output remains faithful to the input's meaning.
- Wasserstein Gradient Flow: We theoretically demonstrate that our training procedure can be viewed as a gradient flow in the Wasserstein space, effectively pushing the model's distribution toward the true data distribution.
Experiments and Results
We evaluated our method across three diverse tasks: machine translation, abstractive summarization, and image captioning.
Machine Translation
Experiments on English-Vietnamese (IWSLT) and English-German (WMT) datasets demonstrate that adding the OT loss ($\mathcal{L}_{seq}$) and the soft-copying loss ($\mathcal{L}_{copy}$) consistently improves performance over the strong GNMT baseline.
Summarization and Image Captioning
On the Gigaword summarization task, our model achieved a ROUGE-L score of 34.0, a significant improvement over the 32.4 MLE baseline. Similarly, in image captioning on MS COCO, the CIDEr score improved from 99.1 to 102.9 for the "Show, Attend & Tell" architecture, indicating better consensus with human-generated captions.
Conclusion
By integrating Optimal Transport into the Seq2Seq training pipeline, we provide a robust mechanism for global semantic supervision. This approach effectively mitigates exposure bias and aligns training objectives with sequence-level evaluation metrics. Our results across multiple NLP domains confirm that combining local syntactic learning (MLE) with global semantic matching (OT) leads to superior sequence generation.
Summary updated