Parallel Interactive Networks for Multi-Domain Dialogue State Generation

Junfan Chen, Richong Zhang, Yongyi Mao, Jie Xu

Introduction

Dialogue State Tracking (DST) serves as the "brain" of modern spoken dialogue systems, maintaining a record of user goals throughout a conversation. In multi-domain scenarios, this task becomes increasingly complex as users frequently transition between topics—for instance, moving from booking a restaurant to arranging transportation. The Parallel Interactive Networks (PIN) model addresses these challenges by explicitly modeling the structured interactions between system and user utterances.

The Challenge: Dependency and Overlap

Existing models often struggle with three primary issues: dependency blindness, slot overlapping, and coarse copying mechanisms. Standard approaches treat dialogue history as a flat sequence, ignoring how system prompts directly shape user responses (in-turn dependency) or how users refer back to previous turns (cross-turn dependency). Furthermore, when multiple domains share similar slots—such as a "price range" appearing in both hotel and restaurant domains—models frequently misattribute values.

Example of cross-turn and in-turn dependencies in a multi-domain dialogue.

The Parallel Interactive Networks (PIN) Model

To address these limitations, the PIN model introduces a structured approach to modeling dialogue dynamics. The architecture is built on four pillars:

1. Interactive Encoder

The core innovation of PIN is its parallel architecture. Instead of processing the entire dialogue history as a single sequence, PIN employs two parallel, double-layer recurrent networks—one for system utterances and one for user utterances.

  • Lower Layer: This layer is designed to capture cross-turn dependencies, modeling how the current system utterance relates to the preceding user context.
  • Higher Layer: This layer focuses on in-turn dependencies, capturing the immediate interaction between the system and user within the current dialogue turn.
The Parallel Interactive Encoder architecture capturing dependencies across turns.

2. Slot-level Context

To mitigate slot overlapping, PIN employs an attention mechanism that generates unique context vectors for every (domain, slot) pair. This ensures that the features used to predict a "Hotel-Price Range" are distinct from those used for "Restaurant-Price Range," significantly reducing domain confusion.

3. Distributed Copy Mechanism

PIN moves beyond indiscriminate copying by calculating three distinct probability distributions: the general vocabulary, the system history, and the user history. This "source-aware" approach allows the model to prioritize information based on its origin, such as favoring system history for entity names suggested by the assistant while relying on user history for specific preferences.

4. Slot Gate

A three-class classifier determines whether a slot is none (not mentioned), dontcare, or requires a specific generated value, streamlining the prediction process and improving efficiency.

Experimental Results

Evaluated on the MultiWOZ 2.0 benchmark, PIN achieved a state-of-the-art Joint Goal Accuracy of 52.44%, outperforming the previous leading model (TRADE) by nearly 4%. Ablation studies confirmed that both the interactive component and the distributed copy mechanism are critical to this success. Notably, PIN demonstrated a ~5% improvement in accuracy for shared slots like "price range," proving its effectiveness in handling domain overlaps.

On the "cleaned" MultiWOZ 2.1 dataset, PIN maintained strong performance with a 48.40% Joint Goal Accuracy. While it trails models that use manual picklists or extra relation information, PIN remains a more generalizable solution that avoids such manual engineering.

Conclusion

The PIN model demonstrates that explicitly modeling the "structured dance" between user and system is vital for robust multi-domain DST. By specializing context at the slot level and implementing source-aware copying, PIN sets a new standard for accuracy and reliability in complex conversational AI, paving the way for more intuitive and context-aware virtual assistants.

Summary updated

BibTeX
@inproceedings{chen2020parallel,
  author = {Junfan Chen and Richong Zhang and Yongyi Mao and Jie Xu},
  title = {{Parallel Interactive Networks for Multi-Domain Dialogue State Generation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.151},
}
Figure