Abstract
For each goal-oriented dialog task of interest, large amounts of data need to be collected for end-to-end learning of a neural dialog system. Collecting that data is a costly and time-consuming process. Instead, we show that we can use only a small amount of data, supplemented with data from a related dialog task. Naively learning from related data fails to improve performance as the related data can be inconsistent with the target task. We describe a meta-learning based method that selectively learns from the related dialog task data. Our approach leads to significant accuracy improvements in an example dialog task.
Introduction
End-to-end (E2E) dialog systems are increasingly popular because they replace complex, hand-engineered pipelines with a single neural network. However, this simplicity comes at a cost: E2E models are notoriously data-hungry. While researchers often have access to data from related tasks (e.g., a hotel booking dataset when building a restaurant reservation system), simply mixing this data in often hurts performance due to "negative transfer."
Negative transfer occurs because related tasks are rarely identical to the target task. They might have different label distributions, conflicting logic, or irrelevant sub-tasks. This research introduces a meta-learning approach that allows models to "borrow" data from related tasks by automatically learning to weight individual examples, effectively filtering out inconsistent information.
Methods: Meta-Learning for Selective Weighting
The proposed meta-learning algorithm treats the weighting of related data as an optimization problem. The training process follows a three-step loop designed to optimize the model's performance on the primary task by selectively leveraging external data:
- Primary Update: The dialog model is updated using a small batch of data from the target (primary) task.
- Related Update: The model is updated using a batch of data from a related task. Crucially, each example in this batch is assigned a weight $w \in (0, 1)$ by a separate "weighting module."
- Meta-Update: The system calculates a meta-gradient to determine how the weighted related examples influenced performance on the primary task. It then updates the weighting module to favor examples that improve primary task performance.
This approach is architecture-agnostic and handles the non-stationarity of training naturally. As shown in the figure above, weights are not static; the model learns to polarize data importance as training progresses, identifying which related examples remain relevant.
Experimental Evaluation
The researchers evaluated their method on two distinct setups:
- Modified MNIST: A sanity check where the primary task is standard MNIST, but the related task has flipped labels for 75% of the data. This tests if the model can identify and ignore "poisoned" data.
- Personalized bAbI Dialog: The primary task involves restaurant reservations requiring user profile personalization (Task 3), while the related task is a standard reservation task without personalization (Task 5). Data scarcity was simulated by limiting the primary task to just 50, 100, or 150 dialogs.
Key Results and Discussion
In the personalized restaurant reservation task, the meta-learning method consistently outperformed all baselines. While standard multi-task learning actually performed worse than using the tiny primary dataset alone (dropping from 54.7% to 51.2% accuracy with 50 dialogs), the proposed method achieved 57.7% accuracy.
The MNIST experiment further validated the model's safety against noise. As illustrated in the weight distribution above, the model successfully learned to ignore poisoned data, achieving 87.86% accuracy compared to approximately 21% for naive methods. This confirms that the weighting module acts as an intelligent filter, recognizing that while different tasks might share general structures (like greetings), their specific domain logic may conflict.
Conclusion
This meta-learning framework provides a robust solution for data-scarce dialog environments by leveraging related datasets without the risk of negative transfer. Future work will focus on scaling this approach to handle multiple related tasks simultaneously and testing on noisier, real-world human-to-human dialog datasets.
Summary updated