Agenda-Based User Simulation for Bootstrapping a POMDP Dialogue System

Jost Schatzmann, Blaise Thomson, Karl Weilhammer, Hui Ye, Steve YoungNAACL 2007

TL;DR

This paper introduces a novel "agenda-based" user simulation method to overcome the challenge of training dialogue systems without initial real-world data. By modeling user intent as a stack of pending actions, this simulator effectively bootstraps a POMDP dialogue manager, leading to a highly competitive dialogue policy that achieves over 90% task completion in real user evaluations.

What Problem Does This Paper Address?

Developing robust dialogue systems, especially those using statistical methods like Partially Observable Markov Decision Processes (POMDPs), typically requires large amounts of in-domain dialogue data for training. However, acquiring this data is a classic "chicken-and-egg" problem: you need a working system to collect data, but you need data to build a working system. Traditional methods like Wizard-of-Oz experiments are time-consuming and may not reflect natural human-computer interactions, while using a handcrafted system to collect data limits the ability to test the statistical DM itself. This paper tackles the challenge of bootstrapping a dialogue manager when such initial data is scarce or non-existent.

Methods

The core contribution is an agenda-based user simulation model. Instead of directly modeling the user's state as a complex set of variables, this approach represents the user's state as a combination of a goal (G) and an agenda (A).

  • Goal (G): This defines what the user wants to achieve, comprising constraints (e.g., "central location," "serves beer") and requests for specific information (e.g., "name," "phone number").
  • Agenda (A): This is a stack-like structure that holds the pending dialogue acts the user needs to perform to achieve their goal. Acts are ordered by priority, with the most urgent at the top.

When a dialogue begins, the goal is randomly generated, and the agenda is populated by converting goal constraints into "inform" acts and goal requests into "request" acts. A "bye" act is added to signal the end. As the dialogue progresses, user acts are selected from the top of the agenda. Machine actions update the agenda by pushing new acts and removing completed ones. This agenda acts as a concise representation of the dialogue history from the user's perspective.

The paper also details how this agenda-based model is used to simulate user behavior for training a POMDP dialogue manager. The state transition model is factorized into goal updates and agenda updates, with the agenda update model being particularly novel in its focus on push-operations triggered by system acts.

Datasets

The paper does not explicitly mention a specific, named dataset for training the simulator itself. Instead, it describes experiments conducted with a prototype HIS dialogue system built for the Tourist Information Domain. This system was then evaluated with 40 human subjects. The dialogues recorded during this evaluation were used to assess the performance of the learned policy.

Key Insights and Ideas

  • Agenda as a User Memory: The agenda acts as a dynamic, prioritized to-do list for the user, effectively capturing their immediate intentions and the flow of the conversation without needing to model complex internal user states.
  • Bootstrapping without Data: The agenda-based simulator, even with manually chosen prior parameters (i.e., without prior training data), proved capable of generating realistic user behavior sufficient to train a competitive dialogue policy. This is a significant breakthrough for systems starting from scratch.
  • Compactness and Simplicity: The agenda-based approach has a very small parameter set, making it amenable to manual initialization and bootstrapping, which is crucial when no training data is available.
  • Realistic Simulation for Policy Learning: The simulated dialogues were realistic enough to train a dialogue policy that performed well when tested with actual human users, demonstrating the simulator's effectiveness.

Key Results

  • The agenda-based simulator was used to train a statistical POMDP-based dialogue manager.
  • The learned dialogue policy, trained solely on simulated data, was evaluated with 40 human subjects.
  • The system achieved a task completion rate of over 90% (145 out of 160 dialogues completed successfully).
  • The average number of turns to completion was 5.59.
  • The evaluation included a mix of native and non-native speakers, suggesting the policy's robustness.

Limitations and Future Work

  • Optimality of Learned Policy: While the learned policy was competitive, the authors acknowledge that it is "not necessarily optimal."
  • Future Work: The authors propose using the data collected from the human subject evaluation to train the simulator itself. This trained simulator could then be used to retrain the DM policy, potentially leading to even better performance. They also suggest further research into optimizing the simulator's parameters.

Summary updated

BibTeX
@inproceedings{schatzmann2007agenda,
  author = {Jost Schatzmann and Blaise Thomson and Karl Weilhammer and Hui Ye and Steve Young},
  title = {{Agenda-Based User Simulation for Bootstrapping a POMDP Dialogue System}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2007},
}
Figure