Meta Back-Translation

Hieu Pham, Xinyi Wang, Yiming Yang, Graham Neubig

Introduction

Neural Machine Translation (NMT) models are notoriously data-hungry. When parallel data—paired sentences in two languages—is scarce, researchers often rely on Back-Translation (BT). This technique involves training a "backward" model to translate monolingual target-language text into the source language, thereby creating "pseudo-parallel" data to train the primary "forward" model.

However, vanilla back-translation suffers from two major flaws. First, the backward model is typically trained once and fixed; if the initial parallel data is poor, the resulting pseudo-data remains poor. Second, NMT faces a "Quality vs. Diversity" paradox. Recent research indicates that the highest-quality translations (those with the highest BLEU scores) from a backward model do not always yield the best forward model. Sometimes, noisier or more diverse data provides a stronger training signal, but there is no clear rule for generating this optimal data.

Meta Back-Translation (MetaBT) addresses these issues by transforming back-translation from a static data-generation step into a dynamic meta-learning process.

The Meta-Learning Framework

MetaBT frames back-translation as a bi-level optimization problem, functioning much like a Teacher-Student relationship:

  • The Student (Forward Model): Learns to translate using the pseudo-parallel data provided by the teacher.
  • The Teacher (Backward Model): Observes how the student performs on a "practice exam" (a held-out meta-validation set). If the student struggles, the teacher adjusts its parameters to generate more helpful pseudo-data in the subsequent round.
Meta Back-Translation Framework

To achieve this, the authors employ a meta-learning framework where the backward model's parameters are updated to minimize the forward model's loss on validation data. Because generating data involves discrete, non-differentiable sampling, the system uses the REINFORCE algorithm (the "log-gradient trick") from reinforcement learning. This allows the gradient of the student's performance to flow back through the sampling process, effectively updating the teacher.

Key Insights

This dynamic approach yields several fascinating behaviors during training:

  • Gradient Matching: The backward model learns to generate data that produces gradients similar to those produced by real, high-quality validation data. It essentially learns to "mimic" the training signal of gold-standard data.
  • Adaptive Difficulty: MetaBT is highly flexible. In large-scale settings where the model might underfit, it generates "easier" data to help the model converge. Conversely, in smaller settings where overfitting is a risk, it generates "harder" or more diverse data to act as a regularizer.
  • Vocabulary Alignment: In multilingual settings, MetaBT automatically learns to favor pseudo-source sentences that utilize the vocabulary of the specific low-resource language being targeted, resisting the drift toward related high-resource languages used during pre-training.

Experimental Results

The researchers evaluated MetaBT on standard benchmarks (WMT En-De ’14 and WMT En-Fr ’14) as well as a multilingual, low-resource TED talk dataset (focusing on Azerbaijani, Belarusian, Glacian, and Slovak).

MetaBT consistently outperformed strong baselines, including standard sampling-based BT and DualNMT:

  • WMT En-De: Achieved a +1.66 BLEU improvement over sampling-based BT, a substantial jump for a highly competitive benchmark.
  • WMT En-Fr: Saw a +0.51 BLEU improvement.
  • Low-Resource: In the multilingual TED talk setting, MetaBT provided gains of up to +1.20 BLEU over the best baselines.
Development BLEU Score Comparison

Furthermore, unlike some joint-training methods that can be numerically unstable, MetaBT demonstrated consistent, stable improvements across different language pairs.

Limitations and Future Work

The primary limitation of MetaBT is its computational overhead. The method requires keeping both the forward and backward models in memory simultaneously, alongside the heavy overhead of calculating second-order gradients (gradients of gradients). Currently, this makes it difficult to scale to the largest "Transformer-Large" models on standard hardware. However, future improvements in automated model parallelism and more powerful hardware will likely allow MetaBT to be applied to even larger architectures.

Summary updated

Abstract

Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data. However, several recent works have found that better translation quality in the pseudo-parallel data does not necessarily lead to a better final translation model, while lower-quality but diverse data often yields stronger results instead. In this paper we propose a new way to generate pseudo-parallel data for back-translation that directly optimizes the final model performance. Specifically, we propose a meta-learning framework where the back-translation model learns to match the forward-translation model's gradients on the development data with those on the pseudo-parallel data. In our evaluations in both the standard datasets WMT En-De'14 and WMT En-Fr'14, as well as a multilingual translation setting, our method leads to significant improvements over strong baselines.
BibTeX
@inproceedings{pham2021meta,
  author = {Hieu Pham and Xinyi Wang and Yiming Yang and Graham Neubig},
  title = {{Meta Back-Translation}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=kc4GjTrCImi},
}
Figure