Abstract
Continual Machine Reading Comprehension (CMRC) aims to incrementally learn from a continuous data stream across time without access to previously seen data. This capability is crucial for real-world MRC systems that must adapt to evolving information. However, learning new domains incrementally often leads to "catastrophic forgetting," where the model loses knowledge of previous domains. This paper introduces MA-MRC, a framework utilizing an uncertainty-aware fixed memory and adversarial domain adaptation. By maintaining a small, strategically updated memory of previous samples and employing adversarial learning to bridge domain gaps, MA-MRC achieves superior performance across multiple benchmarks without the need for full retraining.
Introduction
In standard Machine Reading Comprehension (MRC), models are typically trained on static datasets. However, real-world applications like virtual assistants encounter a shifting stream of data involving new topics and question styles. When these models are fine-tuned on new domains, they suffer from catastrophic forgetting—a rapid decline in performance on earlier tasks. While retraining from scratch is a potential solution, it is computationally prohibitive for large-scale systems.
The challenge of Continual Learning is to enable models to learn incrementally while preserving existing knowledge. MA-MRC addresses this by combining memory-based replay with domain adaptation techniques to ensure that the model's understanding remains stable and generalized across a sequence of tasks.
The MA-MRC Framework
MA-MRC is built upon three primary pillars designed to mitigate forgetting while facilitating the acquisition of new information.
1. Uncertainty-Aware Fixed Memory
Rather than attempting to store all previous data, MA-MRC maintains a fixed-size episodic memory. The key innovation lies in the uncertainty-aware updating strategy. Instead of selecting samples at random, the model uses an entropy-based calculation to identify samples about which it is most "uncertain." The intuition is that samples the model finds difficult or ambiguous are the most likely to be forgotten. By prioritizing these high-uncertainty samples for the memory buffer, the model retains the most informative examples to refresh its knowledge during future training steps.
2. Adversarial Domain Adaptation
Simply replaying old data is often insufficient because the distribution of the new domain may differ significantly from the old one. MA-MRC introduces a domain discriminator that engages in a min-max game with the model's encoder. The encoder aims to produce "domain-invariant" representations, making the memory-stored data and the new domain data indistinguishable to the discriminator. This prevents the model from overfitting to the small subset of memorized examples and helps it find common linguistic patterns across different domains.
3. Knowledge Distillation (KD)
To further anchor the model's knowledge, MA-MRC employs Knowledge Distillation. This ensures that the model’s current predictions on old data remain consistent with its original predictions, providing a regularization effect that prevents the weights from shifting too drastically toward the new domain.
Experimental Evaluation
Datasets and Settings
The authors evaluated MA-MRC under two distinct continual learning settings:
- CDA-C (Corpus-based): A sequence of five diverse MRC datasets with varying passage styles: SQuAD, NaturalQA, HotpotQA, NewsQA, and TriviaQA.
- CDA-Q (Question-based): A split of the SQuAD 1.1 dataset into eight domains based on question types (e.g., "what," "why," "how," "who").
Results
MA-MRC consistently outperformed strong baselines, including regularization-based methods like Elastic Weight Consolidation (EWC) and memory-based methods like DER++.
- CDA-C Performance: MA-MRC achieved an average F1 score of 62.14, significantly outperforming the simple fine-tuning "Lower Bound" (57.03) and exceeding the next best baseline, Online EWC, by nearly 3 points.
- CDA-Q Performance: The model reached an average F1 of 80.41, a substantial improvement over the 67.74 F1 score of the Lower Bound.
- Ablation Study: Removing the adversarial component (w/o Adv) resulted in a drop of approximately 3.5 points in F1 score. This confirms that domain adaptation is critical for generalizing knowledge in a continual learning context.
Conclusion and Future Work
MA-MRC demonstrates that combining uncertainty-driven memory management with adversarial domain adaptation effectively mitigates catastrophic forgetting in MRC tasks. By mimicking the human ability to replay difficult memories and adapt existing knowledge to new contexts, the framework provides a robust path toward more flexible AI systems.
Future research will focus on improving the efficiency of the adversarial training process, which currently introduces a 4-5% overhead per epoch. Additionally, the authors plan to explore more sophisticated sampling strategies and methods for balancing the multiple loss functions (span loss, adversarial loss, and distillation loss) to optimize performance over even longer sequences of domains.
Summary updated