Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System

Chen Chen, Ruizhe Li, Yuchen Hu, Chengwei Qin, Qiang Zhang

Abstract

Intelligent task-oriented dialogue systems (ToDs) are expected to continuously acquire new knowledge, also known as Continual Learning (CL), which is crucial to fit ever-changing user needs. However, catastrophic forgetting dramatically degrades the model performance in face of a long streamed curriculum. In this paper, we aim to overcome the forgetting problem in ToDs and propose a method (HESIT) with hyper-gradient-based exemplar strategy, which samples influential exemplars for periodic retraining. Instead of unilaterally observing data or models, HESIT adopts a profound exemplar selection strategy that considers the general performance of the trained model when selecting exemplars for each task domain. Specifically, HESIT analyzes the training data influence by tracing their hyper-gradient in the optimization process. Furthermore, HESIT avoids estimating Hessian to make it compatible for ToDs with a large pre-trained model. Experimental results show that HESIT effectively alleviates catastrophic forgetting by exemplar selection, and achieves state-of-the-art performance on the largest CL benchmark of ToDs in terms of all metrics.

Introduction

Task-oriented dialogue systems (ToDs) must adapt to new domains—such as flight booking or restaurant reservations—without losing proficiency in previously learned tasks. This requirement defines the challenge of Continual Learning (CL). The primary obstacle in this setting is catastrophic forgetting, where fine-tuning a neural network on new data overwrites weights encoding prior knowledge. In ToDs, this is exacerbated by significant "distributional shifts" between domains, where vocabulary and logic rarely overlap.

While regularization and dynamic architectures offer some relief, they often struggle with long task sequences or introduce unacceptable latency. Rehearsal-based methods, which store a small buffer of old examples for periodic replay, are highly effective. However, the critical challenge lies in determining which specific utterances are most influential and should be preserved in the memory buffer.

Methodology: The HESIT Framework

We propose HESIT (Hyper-gradient-based Exemplar Selection by Influence Tracing), a novel rehearsal-based strategy. Unlike traditional methods that select exemplars based on data statistics (Data-perspective) or model loss (Model-perspective), HESIT adopts a Performance-perspective. It evaluates the "Influence Chain": Data → Model → Performance.

HESIT framework (a) and exemplar selection process (b) illustrating the influence tracing from data to performance.

Influence Tracing and Hyper-gradients

HESIT quantifies the contribution of a training sample by asking: "How would the validation loss change if this specific example were removed from training?" To make this computationally feasible for large pre-trained models like GPT-2, HESIT utilizes Hessian-Free Hyper-gradients. By tracing hyper-gradients recursively through the optimization process, HESIT avoids the prohibitive cost of computing the Hessian matrix.

Exemplar Selection

For each new domain, HESIT identifies the "top-K" most influential examples—those that best support general performance on unseen data—and adds them to a permanent memory buffer. This strategy also helps filter out "noisy" or mislabeled data that would otherwise receive negative influence scores.

Experimental Results

The proposed method was evaluated on a massive 37-domain benchmark derived from TaskMaster 2019/2020, Schema Guided Dialogue (SGD), and MultiWoZ.

Performance and Stability

HESIT achieved state-of-the-art (SOTA) results across all key metrics. In an End-to-End setting, it reached 83.46% Intent Recognition accuracy, significantly outperforming the GSS baseline (81.42%) and random rehearsal (78.22%). Similar gains were observed in Dialogue State Tracking (31.22% JGA) and Natural Language Generation (18.25 BLEU).

Forgetting curve showing HESIT's stable performance compared to other methods over a sequence of 37 tasks.

As shown in the forgetting curve, HESIT maintains high performance even after 30+ tasks, whereas regularization-based methods like EWC and A-GEM experience rapid degradation. This demonstrates that selecting exemplars based on their impact on unseen data performance is far more robust for long-term continual learning.

Conclusion

HESIT effectively mitigates catastrophic forgetting in ToDs by identifying influential exemplars through a scalable, Hessian-free hyper-gradient approach. While the method introduces some initial training overhead and currently focuses on self-domain influence, its ability to maintain high accuracy across dozens of domains marks a significant advancement in building truly adaptive dialogue systems. Future work will explore cross-domain influence to further optimize the memory buffer.

Summary updated

Abstract

Intelligent task-oriented dialogue systems (ToDs) are expected to continuously acquire new knowledge, also known as Continual Learning (CL), which is crucial to fit ever-changing user needs. However, catastrophic forgetting dramatically degrades the model performance in face of a long streamed curriculum. In this paper, we aim to overcome the forgetting problem in ToDs and propose a method (HESIT) with hyper-gradient-based exemplar strategy, which samples influential exemplars for periodic retraining. Instead of unilaterally observing data or models, HESIT adopts a profound exemplar selection strategy that considers the general performance of the trained model when selecting exemplars for each task domain. Specifically, HESIT analyzes the training data influence by tracing their hyper-gradient in the optimization process. Furthermore, HESIT avoids estimating Hessian to make it compatible for ToDs with a large pre-trained model. Experimental results show that HESIT effectively alleviates catastrophic forgetting by exemplar selection, and achieves state-of-the-art performance on the largest CL benchmark of ToDs in terms of all metrics.
BibTeX
@inproceedings{chen2024overcoming,
  author = {Chen Chen and Ruizhe Li and Yuchen Hu and Yuanyuan Chen and Chengwei Qin and Qiang Zhang},
  title = {{Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  url = {https://openreview.net/forum?id=dMOz8yHIpo},
}
Figure