Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions

Shuhao Gu, Bojie Hu, Yang Feng

Abstract

Adapting large-scale pretrained neural machine translation (NMT) models to new domains or languages often results in catastrophic forgetting—the loss of performance on previously learned tasks. While regularization-based methods attempt to mitigate this, they frequently suffer from a "misestimate problem," failing to balance old and new knowledge effectively due to inaccurate quadratic approximations of the loss landscape. This paper introduces a two-stage training framework that identifies "Low Forgetting Risk" (LFR) regions within the parameter space. By constraining updates to these regions, the model can acquire new tasks while preserving existing capabilities without requiring access to original training data or introducing model separation.

Introduction

Neural Machine Translation (NMT) has seen significant advances with the advent of large-scale pretrained models like mBART. However, the computational cost of training these models from scratch makes continual learning—the ability to adapt to new domains or languages sequentially—highly desirable. The primary obstacle is catastrophic forgetting. Conventional approaches like joint training are often impractical due to data privacy or storage constraints, while regularization methods like Elastic Weight Consolidation (EWC) rely on quadratic approximations that often fail to capture the complex geometry of the loss landscape, leading to either excessive forgetting or an inability to learn new information.

The Challenge of Regularization

Regularization-based methods typically add an auxiliary loss term to penalize changes to parameters deemed "important" for previous tasks. The authors argue that these methods suffer from a "misestimate problem." Because they rely on a simplified quadratic approximation of the loss function, they may either over-constrain the model (preventing it from learning the new task) or under-constrain it (leading to forgetting). This paper moves away from these "soft" penalties toward a "hard-constraint" approach based on the local features of the real loss.

Proposed Method: Learning within LFR Regions

The researchers propose a two-stage method to identify and utilize a "safe zone" in the parameter space where updates won't significantly hurt performance on previous tasks.

Stage 1: Searching for LFR Regions

The goal is to find a range $[\theta_{min}, \theta_{max}]$ for each parameter. Two methods are proposed for this search:

  1. Curvature-based Method (LFR-CM): This method utilizes the Fisher Information Matrix to identify "flat" regions in the loss landscape. In these areas, small changes to parameters have a negligible impact on the model's error rate for previous tasks.
  2. Output-based Method (LFR-OM): This approach uses an adversarial strategy. Using a small validation set from the previous task, the model is trained to move its parameters as far as possible from their original values without significantly altering the model's actual translation output (measured via KL divergence). This effectively maps the boundaries of the model's functional stability.

Stage 2: Hard-Constrained Training

Once the LFR regions are defined, the model is trained on the new task (e.g., a new language or domain) using standard cross-entropy loss. However, a hard constraint is applied: parameters are strictly forbidden from moving outside the LFR regions found in Stage 1. This ensures the model remains within the "safe zone" for previous knowledge while utilizing the model's inherent over-parameterization to house new information.

Experimental Evaluation

The method was tested across several challenging scenarios:

  • Domain Adaptation: Adapting German-to-English translation across five diverse domains (IT, Law, Medical, Subtitles, and Koran) using the OPUS dataset.
  • Language Adaptation: Adding support for Greek (an unseen alphabet) and Slovak (similar to Czech) to a model pretrained on 50 languages using FLORES-101.

Key Results

The experimental results demonstrated that the LFR approach provides a superior balance between stability and plasticity:

  • Domain Adaptation: LFR-OM achieved an average BLEU score of 31.09, significantly outperforming standard fine-tuning (26.36) and EWC (30.19).
  • Language Adaptation: LFR-CM proved most effective for adding new languages, maintaining high performance on the original 50 languages while successfully learning Greek and Slovak (25.73 Avg BLEU).
  • Sequential Learning: The method remained robust even when learning multiple tasks in sequence, a scenario where forgetting usually compounds.

Discussion and Insights

The success of the LFR method highlights the importance of leveraging the over-parameterization of large NMT models. These models possess millions of parameters that are not fully utilized for any single task; the LFR method identifies these underused areas and repurposes them for new knowledge. By using hard constraints based on the actual local features of the loss landscape, the method avoids the inaccuracies inherent in traditional regularization approximations.

Conclusion

The proposed two-stage training method offers a robust solution for the continual learning of NMT models. By identifying and operating within Low Forgetting Risk regions, models can adapt to new domains and languages without the need for original training data or the risk of catastrophic forgetting. While there remains a small performance gap compared to task-specific fine-tuning, the method provides a significant step toward generalist translation models that can grow over time.

Summary updated

Abstract

This paper considers continual learning of large-scale pretrained neural machine translation model without accessing the previous training data or introducing model separation. We argue that the widely used regularization-based methods, which perform multi-objective learning with an auxiliary loss, suffer from the misestimate problem and cannot always achieve a good balance between the previous and new tasks. To solve the problem, we propose a two-stage training method based on the local features of the real loss. We first search low forgetting risk regions, where the model can retain the performance on the previous task as the parameters are updated, to avoid the catastrophic forgetting problem. Then we can continually train the model within this region only with the new training data to fit the new task. Specifically, we propose two methods to search the low forgetting risk regions, which are based on the curvature of loss and the impacts of the parameters on the model output, respectively. We conduct experiments on domain adaptation and more challenging language adaptation tasks, and the experimental results show that our method can achieve significant improvements compared with several strong baselines.
BibTeX
@inproceedings{gu2022continual,
  author = {Shuhao Gu and Bojie Hu and Yang Feng},
  title = {{Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  url = {https://openreview.net/forum?id=XiWuGt1nB6},
}
Figure