Conquest—An Open-Source Dialog System for Conferences

Dan Bohus, Sergio Grau Puerto, David Huggins-Daines, Venkatesh Keri, Gopala Krishna, Rohit Kumar, Antoine Raux, Stefanie TomkoNAACL 2007

Sure, here's a blog-style summary of the paper "Conquest—An Open-Source Dialog System for Conferences":

TL;DR

This paper introduces ConQuest, an open-source spoken dialog system designed to provide conference attendees with technical program information. By leveraging the modular Olympus framework, ConQuest aims to lower the barrier for applied research in spoken language interfaces and offers a reusable platform for future conference dialog systems.

What Problem Does This Paper Address?

Developing and deploying spoken language interfaces for real-world applications is a significant undertaking, requiring substantial infrastructure, engineering effort, and domain-specific data. This high cost of entry can hinder applied research in the field. The authors identify the conference domain as an ideal environment for research due to its regular user base, recurring events, and the potential for periodic redeployments and evaluations, offering a common benchmark task for the community.

Methods

ConQuest is built upon the Olympus dialog system framework, which provides a transparent, modular, and open infrastructure. Key components include:

  • Speech Recognition: Utilizes a recognition server with parallel SPHINX-II decoders and a DTMF decoder, employing class-based, state-specific trigram language models. These models were initially trained on an early text-only prototype and later retrained with speech data collected from deployments.
  • Language Understanding: Employs the Phoenix robust parser with a domain-specific shallow semantic grammar, augmented by a domain-independent grammar for generic expressions.
  • Dialog Management: Leverages RavenClaw, a plan-based dialog manager that uses a hierarchical task specification for the conference schedule domain. RavenClaw automatically provides generic conversational skills like error recovery.
  • Backend/Database: A backend agent accesses schedule information from a flat text file database. It handles domain-specific pre-lookup normalization (e.g., author name standardization) and post-lookup processing.
  • Temporal Reference Resolution: An agent resolves temporal expressions into canonical forms.
  • Language Generation: Uses Rosetta, a template-based component, with authoring focused on creating system prompts and information presentation templates.
  • Speech Synthesis: Employs the Cepstral speech synthesis engine with an open-domain unit selection voice, with manual correction of pronunciations for technical terms and names.

The system is designed to be mixed-initiative, allowing users to change the conversation's focus, and features an adaptive information presentation strategy.

Datasets

The paper mentions the collection of corpora from two deployments:

  • Interspeech 2006: A corpus of 174 sessions was collected and is being analyzed.
  • IJCAI 2007: A corpus of 129 sessions was collected and is being transcribed.

These corpora, along with the system's source code, are intended to be freely available. The authors also mention using data from an early text-only prototype to build the initial language model.

Key Insights and Ideas

  • Open-Source for Research: The core idea is to democratize applied research in spoken language interfaces by providing a complete, open-source system and associated data.
  • Conference Domain Suitability: The conference setting offers a natural, recurring environment for testing and evaluating dialog systems with real users.
  • Modularity and Reusability: The reliance on the Olympus framework emphasizes the importance of modularity for adapting the system to different conferences and reducing development effort.
  • Mixed-Initiative Interaction: The system supports mixed-initiative dialog, allowing for more natural and flexible user interactions.
  • Challenges in Real-World Deployment: The authors highlight practical challenges such as handling foreign names, accented characters, text normalization, and ensuring consistency across system resources.
  • Data Collection vs. Expectations: A surprising insight is that the collected dialog corpora were smaller than initially expected, possibly because the information provided was already easily accessible through other means (e.g., paper programs, websites).

Key Results

The paper describes the development and deployment of ConQuest at two conferences: Interspeech 2006 and IJCAI 2007.

  • Interspeech 2006: Involved approximately 7 developers working half-time for three months, with an estimated one man-year of effort. Two systems (desktop and telephone-based) were deployed, collecting 174 sessions of data.
  • IJCAI 2007: Required an estimated two man-months of effort, with three developers working half-time for over a month. This deployment incorporated new features like support for workshops and Indian English speech recognition. It collected 129 sessions of data.

The authors note that the system specifications have been refined, and they expect development time to decrease with further deployments.

Limitations and Future Work

  • Data Collection Volume: The authors acknowledge that the dialog corpora collected were smaller than anticipated. They suggest that combining functionalities with other systems (like DiSCoH) and expanding to provide access to otherwise hard-to-obtain information (e.g., local restaurants, transportation) could increase user engagement and data collection.
  • Future Work: The immediate future involves releasing the collected data, system logs, and source code. They also plan to continue refining the system specifications and expect development time to asymptote to less than a month after more deployments. The potential for combining functionalities with other systems is also mentioned as a future direction.

Summary updated

BibTeX
@inproceedings{bohus2007conquestan,
  author = {Dan Bohus and Sergio Grau Puerto and David Huggins-Daines and Venkatesh Keri and Gopala Krishna and Rohit Kumar and Antoine Raux and Stefanie Tomko},
  title = {{Conquest—An Open-Source Dialog System for Conferences}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2007},
}
Figure