A Discriminative Classification-Based Approach to Information State Updates for a Multi-Domain Dialog System

Dilek Hakkani-Tür, Gökhan Tür, Larry P. Heck, Ashley Fidler, Asli ÇelikyilmazInterspeech 2012 · OpenReview

Introduction

In goal-oriented dialog systems, maintaining an accurate "Information State" (IS) is critical. The IS represents the system's current understanding of user intent, domain, and constraints (slots). As conversations progress, the system must decide how to integrate new utterances with existing context—a process known as the Information State update. Traditional rule-based approaches are often brittle and difficult to scale across multiple domains. This paper presents a discriminative classification approach using AdaBoost to manage these updates, achieving significant performance gains in complex, multi-turn interactions.

System Architecture

The proposed framework treats the Information State update as a classification problem rather than a set of manual rules. As shown in the system architecture, the Spoken Language Understanding (SLU) component processes user input to produce an initial interpretation ($I_i$). This interpretation, along with the dialog history ($S_{1...i-1}$), is fed into the Information State Update module.

System architecture for information state updates

The update module determines the optimal transition to the next state ($S_i$), which then informs the Dialog Flow and subsequent system actions ($A_i$).

Methodology: Update Actions and Classification

The core of the approach is a classifier trained to select from 12 distinct "update actions." These actions define how new information interacts with the current state. For instance, an "Initiate" action starts a new task, while an "Override-slot" action replaces an existing constraint with new information provided by the user.

Mapping user turns to update actions and information states

Feature Engineering

The classifier utilizes two primary feature sets:

  1. Lexical Features: Word n-grams from the user's utterance. Phrases like "how about" or "instead" serve as strong indicators of context-dependent updates.
  2. SLU-derived Features: These features compare the current SLU output with the previous Information State. By checking for changes in domain, intent, or conflicting slot values, the model can determine the relationship between turns without domain-specific hard-coding.

The researchers also explored "Context Independent" (CI) vs. "Context Dependent" (CD) tagging. In the CI approach, vague utterances (e.g., "How about tomorrow?") are tagged as vague by the SLU, leaving the Dialog Manager to resolve the domain based on history. Interestingly, the CI approach resulted in slightly lower error rates.

Experimental Setup and Results

The model was evaluated on a dataset of 2,638 user turns across 274 sessions, covering 25 different domains. The SLU models themselves were trained on 20,000 utterances. The researchers compared the discriminative model against a lexical baseline and a simple "Initiate-only" baseline.

Feature Set Error Rate (CI)
Baseline (Initiate-only) 34.3%
Word n-grams only 27.5%
Full Model (n-grams + SLU Features) 21.5%

The full model achieved a 20% relative error reduction over the lexical-only baseline. A key finding was that domain-independent features—those focusing on the relationship between turns rather than specific values—allowed the system to remain scalable and "seamless" across diverse topics like weather and flight booking.

Conclusion

By framing Information State updates as a discriminative classification task, this approach provides a robust and scalable alternative to traditional dialog management. The use of AdaBoost with a combination of lexical and relational SLU features allows the system to handle the inherent vagueness of natural conversation while significantly reducing state-tracking errors. Future work will explore incorporating uncertainty from speech recognition and leveraging implicit user feedback through reinforcement learning.

Summary updated

BibTeX
@inproceedings{hakkanitur2012discriminative,
  author = {Dilek Hakkani-Tür and Gökhan Tür and Larry P. Heck and Ashley Fidler and Asli Çelikyilmaz},
  title = {{A Discriminative Classification-Based Approach to Information State Updates for a Multi-Domain Dialog System}},
  booktitle = {Interspeech},
  year = {2012},
  url = {https://openreview.net/forum?id=zGNKzcnCqf},
}
Figure