Abstract
Adapting large-scale pretrained neural machine translation (NMT) models to new domains or languages often results in catastrophic forgetting—the loss of performance on previously learned tasks. While regularization-based methods attempt to mitigate this, they frequently suffer from a "misestimate problem," failing to balance old and new knowledge effectively due to inaccurate quadratic approximations of the loss landscape. This paper introduces a two-stage training framework that identifies "Low Forgetting Risk" (LFR) regions within the parameter space. By constraining updates to these regions, the model can acquire new tasks while preserving existing capabilities without requiring access to original training data or introducing model separation.
Introduction
Neural Machine Translation (NMT) has seen significant advances with the advent of large-scale pretrained models like mBART. However, the computational cost of training these models from scratch makes continual learning—the ability to adapt to new domains or languages sequentially—highly desirable. The primary obstacle is catastrophic forgetting. Conventional approaches like joint training are often impractical due to data privacy or storage constraints, while regularization methods like Elastic Weight Consolidation (EWC) rely on quadratic approximations that often fail to capture the complex geometry of the loss landscape, leading to either excessive forgetting or an inability to learn new information.
The Challenge of Regularization
Regularization-based methods typically add an auxiliary loss term to penalize changes to parameters deemed "important" for previous tasks. The authors argue that these methods suffer from a "misestimate problem." Because they rely on a simplified quadratic approximation of the loss function, they may either over-constrain the model (preventing it from learning the new task) or under-constrain it (leading to forgetting). This paper moves away from these "soft" penalties toward a "hard-constraint" approach based on the local features of the real loss.
Proposed Method: Learning within LFR Regions
The researchers propose a two-stage method to identify and utilize a "safe zone" in the parameter space where updates won't significantly hurt performance on previous tasks.
Stage 1: Searching for LFR Regions
The goal is to find a range $[\theta_{min}, \theta_{max}]$ for each parameter. Two methods are proposed for this search:
- Curvature-based Method (LFR-CM): This method utilizes the Fisher Information Matrix to identify "flat" regions in the loss landscape. In these areas, small changes to parameters have a negligible impact on the model's error rate for previous tasks.
- Output-based Method (LFR-OM): This approach uses an adversarial strategy. Using a small validation set from the previous task, the model is trained to move its parameters as far as possible from their original values without significantly altering the model's actual translation output (measured via KL divergence). This effectively maps the boundaries of the model's functional stability.
Stage 2: Hard-Constrained Training
Once the LFR regions are defined, the model is trained on the new task (e.g., a new language or domain) using standard cross-entropy loss. However, a hard constraint is applied: parameters are strictly forbidden from moving outside the LFR regions found in Stage 1. This ensures the model remains within the "safe zone" for previous knowledge while utilizing the model's inherent over-parameterization to house new information.
Experimental Evaluation
The method was tested across several challenging scenarios:
- Domain Adaptation: Adapting German-to-English translation across five diverse domains (IT, Law, Medical, Subtitles, and Koran) using the OPUS dataset.
- Language Adaptation: Adding support for Greek (an unseen alphabet) and Slovak (similar to Czech) to a model pretrained on 50 languages using FLORES-101.
Key Results
The experimental results demonstrated that the LFR approach provides a superior balance between stability and plasticity:
- Domain Adaptation: LFR-OM achieved an average BLEU score of 31.09, significantly outperforming standard fine-tuning (26.36) and EWC (30.19).
- Language Adaptation: LFR-CM proved most effective for adding new languages, maintaining high performance on the original 50 languages while successfully learning Greek and Slovak (25.73 Avg BLEU).
- Sequential Learning: The method remained robust even when learning multiple tasks in sequence, a scenario where forgetting usually compounds.
Discussion and Insights
The success of the LFR method highlights the importance of leveraging the over-parameterization of large NMT models. These models possess millions of parameters that are not fully utilized for any single task; the LFR method identifies these underused areas and repurposes them for new knowledge. By using hard constraints based on the actual local features of the loss landscape, the method avoids the inaccuracies inherent in traditional regularization approximations.
Conclusion
The proposed two-stage training method offers a robust solution for the continual learning of NMT models. By identifying and operating within Low Forgetting Risk regions, models can adapt to new domains and languages without the need for original training data or the risk of catastrophic forgetting. While there remains a small performance gap compared to task-specific fine-tuning, the method provides a significant step toward generalist translation models that can grow over time.
Summary updated