Continual Machine Reading Comprehension via Uncertainty-aware Fixed Memory and Adversarial Domain Adaptation

Zhijing Wu, Hua Xu, Jingliang Fang, Kai Gao

Abstract

Machine Reading Comprehension (MRC) models typically struggle with catastrophic forgetting when trained on a sequential stream of new domains. We present MA-MRC (Memory-Adversarial MRC), a framework designed for continual learning in MRC tasks. MA-MRC utilizes an uncertainty-aware fixed memory to retain critical samples from previous domains, combined with adversarial domain adaptation to extract domain-invariant features. Experimental results on corpus-based (CDA-C) and question-type-based (CDA-Q) settings demonstrate that MA-MRC significantly outperforms existing continual learning baselines, maintaining high performance across diverse datasets without the need for full retraining.

Introduction

Standard MRC models are often trained on static datasets like SQuAD. However, real-world applications such as intelligent assistants and search engines frequently encounter evolving data distributions. Fine-tuning a pre-trained model on a new domain often leads to "catastrophic forgetting," where the model's performance on previously learned tasks degrades sharply. While retraining on the entire accumulated dataset is a possible solution, it is computationally prohibitive as data grows. Continual learning aims to address this by enabling models to learn incrementally from a data stream while preserving prior knowledge.

Methodology: The MA-MRC Framework

The proposed MA-MRC framework balances the acquisition of new knowledge with the retention of old skills through three primary components:

The MA-MRC architecture for initial training and incremental learning.

1. Uncertainty-aware Fixed Memory

Rather than employing simple random sampling, MA-MRC uses an entropy-based strategy to select samples for its fixed-size memory. By prioritizing "uncertain" samples—those where the model's prediction confidence is lower—the framework ensures that the most informative and potentially forgettable examples are preserved. This targeted retention provides a stronger signal for maintaining performance on older domains compared to random buffers.

2. Adversarial Domain Adaptation

To prevent the model from overfitting to the limited samples in memory, an adversarial training component is introduced. A domain discriminator is trained to identify whether features originate from the current domain or the memory of previous domains. Simultaneously, the MRC model (the generator) is optimized to produce features that are domain-neutral, effectively "fooling" the discriminator. This process encourages the extraction of generalizable features that transcend specific domain quirks.

3. Knowledge Distillation (KD)

During incremental learning, MA-MRC employs knowledge distillation to maintain stability. The model is penalized if its output logits on memory samples deviate significantly from those produced by the previous iteration of the model. This "soft" constraint ensures that the model's decision boundaries for old tasks remain relatively stable while adapting to new data.

Experimental Evaluation

The authors evaluated MA-MRC under two distinct continual learning scenarios:

  • CDA-C (Corpus-based): Sequential training across five major MRC datasets: SQuAD 1.1, NaturalQA, HotpotQA, NewsQA, and TriviaQA.
  • CDA-Q (Question-type-based): Training on SQuAD partitioned into eight domains based on question words (e.g., "what", "how", "why").

Key Results

MA-MRC demonstrated superior performance over strong baselines, including Elastic Weight Consolidation (EWC) and Dark Experience Replay (DER++).

Performance comparison of MA-MRC against baselines under the CDA-Q setting.

In the CDA-C setting, MA-MRC achieved an average F1 score of 60.61, outperforming EWC (53.85) and DER++ (57.61). In the CDA-Q setting, the model reached an average F1 of 74.15, a 12-point improvement over simple fine-tuning. Ablation studies confirmed that both the adversarial component and knowledge distillation are critical; removing either led to a measurable drop in performance, highlighting the necessity of both "remembering" and "adapting."

Discussion and Conclusion

The success of MA-MRC underscores several key insights in continual learning for NLP. First, not all memories are equal; focusing on high-entropy samples provides a more efficient use of limited memory capacity. Second, adversarial learning is a powerful tool for generalization, preventing the model from merely memorizing a small subset of old data. Finally, the combination of stability (via KD) and plasticity (via adversarial adaptation) allows for effective navigation of the stability-plasticity trade-off.

Future work may involve exploring more sophisticated sampling strategies beyond entropy and investigating methods to mitigate the computational overhead of adversarial training, which currently adds approximately 4-5% to training time.

Summary updated

Abstract

Continual Machine Reading Comprehension aims to incrementally learn from a continuous data stream across time without access the previous seen data, which is crucial for the development of real-world MRC systems. However, it is a great challenge to learn a new domain incrementally without catastrophically forgetting previous knowledge. In this paper, MA-MRC, a continual MRC model with uncertainty-aware fixed Memory and Adversarial domain adaptation, is proposed. In MA-MRC, a fixed size memory stores a small number of samples in previous domain data along with an uncertainty-aware updating strategy when new domain data arrives. For incremental learning, MA-MRC not only keeps a stable understanding by learning both memory and new domain data, but also makes full use of the domain adaptation relationship between them by adversarial learning strategy. The experimental results show that MA-MRC is superior to strong baselines and has a substantial incremental learning ability without catastrophically forgetting under two different continual MRC settings.
BibTeX
@inproceedings{wu2022continual,
  author = {Zhijing Wu and Hua Xu and Jingliang Fang and Kai Gao},
  title = {{Continual Machine Reading Comprehension via Uncertainty-aware Fixed Memory and Adversarial Domain Adaptation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.findings-naacl.179},
}
Figure