Improving Sequence-to-Sequence Learning via Optimal Transport

Liqun Chen, Yizhe Zhang, Ruiyi Zhang, Chenyang Tao, Zhe Gan, Haichao Zhang, Bai Li, Dinghan Shen, Changyou Chen, Lawrence Carin

Abstract

Sequence-to-sequence models are commonly trained via maximum likelihood estimation (MLE). However, standard MLE training considers a word-level objective, predicting the next word given the previous ground-truth partial sentence. This procedure focuses on modeling local syntactic patterns, and may fail to capture long-range semantic structure. We present a novel solution to alleviate these issues. Our approach imposes global sequence-level guidance via new supervision based on optimal transport, enabling the overall characterization and preservation of semantic features. We further show that this method can be understood as a Wasserstein gradient flow trying to match our model to the ground truth sequence distribution. Extensive experiments are conducted to validate the utility of the proposed approach, showing consistent improvements over a wide variety of NLP tasks, including machine translation, abstractive text summarization, and image captioning.

Introduction

Sequence-to-sequence (Seq2Seq) models have revolutionized natural language processing, powering applications from machine translation to text summarization. Despite their success, the standard training paradigm—Maximum Likelihood Estimation (MLE)—suffers from two significant drawbacks: exposure bias and metric mismatch. Exposure bias arises because models are trained on ground-truth prefixes but must generate sequences from their own potentially erroneous predictions during inference. Metric mismatch occurs because MLE optimizes for local, position-specific word accuracy, whereas evaluation metrics like BLEU or ROUGE prioritize global semantic coherence.

To address these challenges, we propose a novel training objective that incorporates Optimal Transport (OT). By treating sentences as distributions of semantic embeddings, OT provides a global, sequence-level supervision signal that complements the local syntactic focus of MLE.

Methodology

The proposed framework augments the standard MLE loss with an OT-based loss ($\mathcal{L}_{ot}$). Unlike MLE, which requires exact word-for-word alignment, OT calculates the minimum "cost" to transform the model's generated sequence distribution into the target sequence distribution in an embedding space.

Optimal Transport Framework

The core of our approach is the computation of the Wasserstein distance between the generated and target sequences. To ensure the objective is differentiable and computationally efficient, we employ the Inexact Proximal point method for Optimal Transport (IPOT). This algorithm provides a stable and efficient way to compute the transport plan $T$ and the cost matrix $C$.

The proposed Seq2Seq training framework using Optimal Transport. Feature vectors S and S' are used to compute a cost matrix C, which is then processed by the IPOT algorithm to generate a transport plan T, ultimately yielding the OT loss.

Technical Innovations

  1. Soft-argmax: To bridge the gap between discrete word selection and differentiable optimization, we use a soft-argmax operator to represent the model's belief over the vocabulary.
  2. Soft-copying Mechanism: We introduce an additional OT loss between the generated output and the source input. This acts as a semantic "copy" mechanism, ensuring the output remains faithful to the input's meaning.
  3. Wasserstein Gradient Flow: We theoretically demonstrate that our training procedure can be viewed as a gradient flow in the Wasserstein space, effectively pushing the model's distribution toward the true data distribution.

Experiments and Results

We evaluated our method across three diverse tasks: machine translation, abstractive summarization, and image captioning.

Machine Translation

Experiments on English-Vietnamese (IWSLT) and English-German (WMT) datasets demonstrate that adding the OT loss ($\mathcal{L}_{seq}$) and the soft-copying loss ($\mathcal{L}_{copy}$) consistently improves performance over the strong GNMT baseline.

Performance comparison of GNMT baselines against the proposed OT-augmented models (GNMT + L_seq + L_copy) on various translation tasks.

Summarization and Image Captioning

On the Gigaword summarization task, our model achieved a ROUGE-L score of 34.0, a significant improvement over the 32.4 MLE baseline. Similarly, in image captioning on MS COCO, the CIDEr score improved from 99.1 to 102.9 for the "Show, Attend & Tell" architecture, indicating better consensus with human-generated captions.

Conclusion

By integrating Optimal Transport into the Seq2Seq training pipeline, we provide a robust mechanism for global semantic supervision. This approach effectively mitigates exposure bias and aligns training objectives with sequence-level evaluation metrics. Our results across multiple NLP domains confirm that combining local syntactic learning (MLE) with global semantic matching (OT) leads to superior sequence generation.

Summary updated

Abstract

Sequence-to-sequence models are commonly trained via maximum likelihood estimation (MLE). However, standard MLE training considers a word-level objective, predicting the next word given the previous ground-truth partial sentence. This procedure focuses on modeling local syntactic patterns, and may fail to capture long-range semantic structure. We present a novel solution to alleviate these issues. Our approach imposes global sequence-level guidance via new supervision based on optimal transport, enabling the overall characterization and preservation of semantic features. We further show that this method can be understood as a Wasserstein gradient flow trying to match our model to the ground truth sequence distribution. Extensive experiments are conducted to validate the utility of the proposed approach, showing consistent improvements over a wide variety of NLP tasks, including machine translation, abstractive text summarization, and image captioning.
BibTeX
@inproceedings{chen2019improving,
  author = {Liqun Chen and Yizhe Zhang and Ruiyi Zhang and Chenyang Tao and Zhe Gan and Haichao Zhang and Bai Li and Dinghan Shen and Changyou Chen and Lawrence Carin},
  title = {{Improving Sequence-to-Sequence Learning via Optimal Transport}},
  booktitle = {International Conference on Learning Representations},
  year = {2019},
  url = {https://openreview.net/forum?id=skQUGBtuew},
}
Figure