Parallel Interactive Networks for Multi-Domain Dialogue State Generation

Junfan Chen, Richong Zhang, Yongyi Mao, Jie Xu

Abstract

Multi-Domain Dialogue State Tracking (MDST) is a critical component in task-oriented dialogue systems, requiring the tracking of user goals across multiple domains. Traditional models often overlook the interactive nature of dialogue and struggle with overlapping slots between domains. This paper introduces Parallel Interactive Networks (PIN), a novel architecture that explicitly models dependencies between system and user utterances through an interactive encoder and a distributed copy mechanism. PIN achieves state-of-the-art performance on the MultiWOZ 2.0 benchmark, demonstrating the effectiveness of separating system and user histories.

Introduction: The Challenges of Multi-Domain Tracking

In complex multi-domain dialogues—such as booking a hotel followed by a restaurant and a taxi—the system must accurately maintain a "dialogue state" consisting of (domain, slot, value) triplets. Current MDST research identifies two primary gaps:

  1. Ignored Dependencies: Dialogue is inherently interactive. A user's response often depends on the system's previous prompt (in-turn dependency), and later requests may refer back to earlier system suggestions (cross-turn dependency). Treating dialogue history as a flat sequence fails to capture these nuances.
  2. Slot Overlapping: Many domains share identical slots (e.g., 'price range' in both 'hotel' and 'restaurant'). Without specialized features, models frequently "leak" values across domains, leading to incorrect state updates.
Figure 1: Illustration of in-turn and cross-turn dependencies in a multi-domain dialogue.

Methodology: Parallel Interactive Networks (PIN)

The PIN model addresses these challenges through three architectural innovations designed to mimic the structure of human conversation.

Interactive Encoder

Instead of a single recurrent neural network (RNN), PIN employs two parallel hierarchical networks—one for system utterances and one for user utterances. These networks interact at each layer, allowing the model to jointly capture how system suggestions influence user goals and vice versa.

Slot-level Context

To mitigate the overlapping slot problem, PIN utilizes an attention mechanism to extract specific features for every (domain, slot) pair. This ensures the model focuses on relevant context for a "hotel price range" without interference from "restaurant" mentions.

Distributed Copy Mechanism

While standard models use a single pointer-generator to copy words from the entire history, PIN recognizes that different types of information originate from different sources. It employs two separate "copy heads" to selectively pull information from either system or user history, governed by a learnable gate.

Figure 4: The overall architecture of PIN, featuring the interactive encoder, slot-level attention, and distributed copy mechanism.

Evaluation and Results

The researchers evaluated PIN on the MultiWOZ 2.0 and MultiWOZ 2.1 benchmarks.

  • Performance: On MultiWOZ 2.0, PIN achieved a Joint Goal Accuracy of 52.44%, significantly outperforming the previous state-of-the-art and the widely used TRADE model (48.62%).
  • Ablation Studies: Removing the interactive component of the encoder led to a noticeable drop in performance, confirming that cross-talk between system and user paths is vital for accuracy.
  • Error Analysis: Qualitative reviews showed that PIN makes fewer errors in the "hotel" and "restaurant" domains compared to baselines, primarily due to its superior handling of dependencies between system suggestions and user confirmations.

Conclusion and Future Work

PIN demonstrates that explicitly modeling the distinct roles of the system and user significantly improves dialogue state tracking. By processing these roles in parallel and using a distributed copy mechanism, the model reduces cross-domain confusion and better handles complex dependencies.

Future research directions include incorporating explicit slot relationships—as seen in models like SST—and improving the computational efficiency of the parallel architecture. Additionally, exploring the zero-shot transfer capabilities of this interactive structure remains a promising area for investigation.

Summary updated

Abstract

The dependencies between system and user utterances in the same turn and across different turns are not fully considered in existing multidomain dialogue state tracking (MDST) models. In this study, we argue that the incorporation of these dependencies is crucial for the design of MDST and propose Parallel Interactive Networks (PIN) to model these dependencies. Specifically, we integrate an interactive encoder to jointly model the in-turn dependencies and cross-turn dependencies. The slot-level context is introduced to extract more expressive features for different slots. And a distributed copy mechanism is utilized to selectively copy words from historical system utterances or historical user utterances. Empirical studies demonstrated the superiority of the proposed PIN model.
BibTeX
@inproceedings{chen2020parallel,
  author = {Junfan Chen and Richong Zhang and Yongyi Mao and Jie Xu},
  title = {{Parallel Interactive Networks for Multi-Domain Dialogue State Generation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.151},
}
Figure