A Flexible Conversational Dialog System for MP3 Player

Qi Zhang (张奇), Fuliang Weng, Lawrence Cavedon, Badri Raghunathan, Danilo Mirkovic, Ben Bei, Heather Pon-Barry, Harry Bratt, Hua Cheng, Hauke Schmidt, Rohit Mishra, Brian Lathrop, Qi Zhang, Tobias Scheideck, Kui Xu, Tess Hand-Bender, Stanley Peters, Liz Shriberg, Carsten BergmannEMNLP 2005

Abstract

In the high-cognitive-load environment of a vehicle, traditional voice command systems often prove brittle and distracting. This paper presents a flexible, multi-threaded conversational system developed by researchers from Bosch, Stanford, SRI, and VW. The system moves beyond rigid keyword-based interfaces to handle natural language phenomena such as context-dependent references, verbal revisions, and simultaneous multi-topic management. By employing a Java-based event-driven middleware and an ontology-driven knowledge base, the system achieves a more natural and robust interaction model for in-car device control.

Introduction

Interacting with in-car devices while driving poses significant safety risks. While voice interfaces aim to mitigate distraction, legacy systems often require specific keywords and fail to handle the nuances of natural conversation. If a user changes their mind or is interrupted, these systems frequently break down. To address these challenges, the authors developed a "flexible" dialog system capable of handling interruptions, understanding natural context (e.g., resolving pronouns like "it"), and managing information overload through intelligent content optimization.

System Architecture

The system departs from the traditional "hub-and-spoke" architectures of the early 2000s, utilizing a Java-based, event-driven middleware. This design facilitates dynamic communication between various specialized modules, allowing for a more fluid interaction flow.

Overview of the flexible conversational dialog system architecture.

Key technical components include:

  • Dual-Path NLU: The Natural Language Understanding (NLU) module performs deep linguistic analysis to produce semantic structures. In parallel, it uses a topic classifier to provide a "best guess" if the deep analysis fails, ensuring system robustness.
  • Multi-threaded Dialogue Manager (DM): Using a "dialogue-move" approach, the DM maintains a history of the conversation. This allows it to resolve pronouns (anaphora) and manage multiple concurrent conversation threads, such as jumping between music selection and navigation.
  • Knowledge Management via OWL: Domain knowledge is stored using the Web Ontology Language (OWL). This makes the system highly portable; to move from an MP3 player to a GPS, developers theoretically only need to swap the ontology.
  • Content Optimizer: This acts as an intelligent filter between the database and the user. If a search returns too many results (e.g., 50 songs), the optimizer uses the ontology to suggest ways to narrow the search, keeping the driver's attention on the road.

Key Features and Insights

The system incorporates several advanced features to improve the user experience in a mobile environment:

  • Prosody-Aware End-pointing: The system uses "prosodic cues" (the rhythm and pitch of speech) to distinguish between a driver pausing to think and a driver who has actually finished their sentence.
  • The Power of Multi-threading: One of the most human-like features is the ability to diverge. A user can start a music search, ask "Where am I?", and then return to the music search seamlessly without losing their place.
  • Verbal Revision: The system is designed to handle "on-the-fly" corrections. If a user says "Play 'Bohemian Rhapsody'... no, wait, play 'Killer Queen'," the DM recognizes the revision rather than getting confused by conflicting titles.

Evaluation and Results

The system was evaluated on a custom set of 11 specific tasks within the MP3 player domain, ranging from simple playback requests to complex database queries involving multiple constraints. In early evaluations, the system achieved a task completion rate of 80%. While 80% might seem low by modern standards, for a 2005-era system handling complex, multi-threaded natural language in a noisy car environment, it represented significant progress toward a more natural user interface.

Conclusion and Future Work

While the current evaluation is limited to a small set of tasks, the use of W3C standards like OWL and Protégé facilitates easier porting to other in-car systems. The authors acknowledge that while the system is portable, expansion into domains like restaurant selection remains a work in progress. Future work will focus on further integrating prosodic features into the NLU and Dialogue Manager to improve performance and robustness in real-world driving scenarios.

Summary updated

BibTeX
@inproceedings{2005flexible,
  author = {Qi Zhang (张奇) and Fuliang Weng and Lawrence Cavedon and Badri Raghunathan and Danilo Mirkovic and Ben Bei and Heather Pon-Barry and Harry Bratt and Hua Cheng and Hauke Schmidt and Rohit Mishra and Brian Lathrop and Qi Zhang and Tobias Scheideck and Kui Xu and Tess Hand-Bender and Stanley Peters and Liz Shriberg and Carsten Bergmann},
  title = {{A Flexible Conversational Dialog System for MP3 Player}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
}
Figure