Abstract
Intelligent task-oriented dialogue systems (ToDs) are expected to continuously acquire new knowledge—a process known as Continual Learning (CL)—to adapt to evolving user needs. However, neural networks often suffer from catastrophic forgetting, where performance on previously learned tasks degrades sharply as new ones are introduced. This paper introduces HESIT (Hyper-gradient-based Exemplar Selection by Influence Tracing), a novel rehearsal-based strategy that identifies and preserves influential training examples for periodic retraining. By tracing hyper-gradients in the optimization process, HESIT selects exemplars that maximize general model performance while avoiding the computational burden of Hessian estimation. Experimental results on a large-scale 37-domain benchmark demonstrate that HESIT achieves state-of-the-art performance across all metrics, effectively mitigating catastrophic forgetting in ToDs.
Introduction
Modern virtual assistants must handle a diverse and growing array of tasks, from booking flights to managing smart home devices. To remain useful, these systems must learn new domains sequentially. However, the phenomenon of catastrophic forgetting presents a significant hurdle: as the model updates its weights for a new domain (e.g., "Alarm Clock"), it often overwrites the parameters essential for a previously mastered domain (e.g., "Hotel Booking").
Existing approaches to this problem generally fall into three categories, each with distinct drawbacks:
- Regularization: Methods like Elastic Weight Consolidation (EWC) constrain weight updates but lose efficacy as the number of tasks grows.
- Dynamic Architectures: Using adapters or modular components avoids forgetting but increases inference latency and requires complex logic to route user queries to the correct module.
- Standard Rehearsal: Storing a subset of old data (exemplars) to replay during new training is effective but often relies on random or diversity-based selection, which may not capture the most "influential" data points for maintaining performance.
The HESIT Framework
HESIT addresses these limitations through a performance-oriented exemplar selection strategy. Rather than selecting data based on simple diversity or model loss, HESIT evaluates the influence of specific data points on the model’s ability to generalize to unseen data.
Hyper-gradient Influence Tracing
The core of HESIT is the "Influence Chain": Data → Model → Performance. The method asks a fundamental question: "If this specific training sample were included, how much would it improve the model's accuracy on a validation set?" To answer this, the authors trace the hyper-gradient—the derivative of the validation loss with respect to the weight of a training point.
Hessian-Free Optimization for Large Models
Traditional influence functions require calculating the inverse Hessian matrix, a process that is computationally prohibitive and numerically unstable for large-scale pre-trained models like GPT-2. HESIT overcomes this by utilizing a recursive update strategy that bypasses the Hessian entirely. This "Hessian-free" approach allows the system to remain compatible with the large transformer-based architectures common in modern ToDs.
Top-K Selection Strategy
After training on a new domain, HESIT scores every sample based on its calculated influence. The system then retains only the "Top-K" most influential samples in a small, fixed-size memory buffer. These exemplars are replayed during subsequent training cycles to anchor the model’s knowledge of previous domains.
Experimental Setup and Results
The researchers evaluated HESIT using the most comprehensive continual learning benchmark for ToDs to date. This benchmark aggregates four major datasets—TaskMaster 2019, TaskMaster 2020, Schema Guided Dialogue (SGD), and MultiWoZ—comprising a curriculum of 37 distinct domains and over 31,000 dialogues.
Performance Metrics
The study measured performance across three critical modules: Intent Recognition, Dialogue State Tracking (DST), and Natural Language Generation (NLG). HESIT achieved State-of-the-Art (SOTA) results in both modularized and end-to-end (E2E) settings:
- Intent Accuracy: Reached 83.46% in the E2E setting, outperforming standard random selection by approximately 5%.
- Dialogue State Tracking: Achieved a Joint Goal Accuracy (JGA) of 31.22% (E2E) and 40.04% (Modular), consistently surpassing other advanced selection strategies like GSS and ARPER.
Data Quality and Filtering
An interesting byproduct of the HESIT approach is its ability to identify "detrimental" data. The authors observed that samples with the lowest influence scores often contained noisy history, incorrect API calls, or labeling errors. By naturally excluding these samples from the replay buffer, HESIT ensures that the model is not "re-learning" mistakes from the past.
Discussion and Future Work
HESIT demonstrates that for long-sequence continual learning, performance-based data rehearsal is more robust than weight regularization. By focusing on the samples that most effectively represent the underlying logic of a domain, HESIT maintains high accuracy even after 37 consecutive task shifts.
However, some challenges remain. Currently, HESIT evaluates influence based on a sample's own domain validation set. Future research could explore cross-domain influence—identifying samples in one domain (e.g., "Hotels") that might provide positive transfer to a related domain (e.g., "Travel Planning"). Additionally, while the Hessian-free approach is efficient, the "reproducible retraining" step required for gradient tracing still introduces more computational overhead than simple random sampling, suggesting a trade-off between training time and long-term model stability.
Conclusion
HESIT provides a scalable and effective solution for the catastrophic forgetting problem in task-oriented dialogue systems. By shifting the focus from what data is available to which data is most influential for performance, HESIT sets a new standard for rehearsal-based continual learning in complex, multi-domain environments.
Summary updated