Bootstrapping Spoken Dialog Systems with Data Reuse

Giuseppe Di Fabbrizio, Gökhan Tür, Dilek Hakkani-TürSIGDIAL Workshop 2004 · OpenReview

Authors: Guiseppe Di Fabbrizio, Gokhan Tur, Dilek Hakkani-Tür
Source: Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue at HLT-NAACL 2004.

2004.

Abstract

Building a Spoken Dialog System (SDS) from scratch usually requires a mountain of hand-labeled data and manual design. This paper introduces a "bootstrapping" framework that recycles data, templates, and models from existing applications to launch a functional system for a new domain with zero application-specific training data.

Introduction

The "cold start" problem is a significant hurdle in the development of Spoken Dialog Systems. Traditionally, creating a system for a specific domain—such as pharmaceutical customer care—requires developers to manually define intents (call-types), collect and transcribe thousands of utterances, and hand-label data to train Automatic Speech Recognition (ASR) and Spoken Language Understanding (SLU) models. This process is resource-intensive and difficult to scale. This research demonstrates that by reusing assets from previous projects and generic language patterns, developers can bypass the initial data-collection phase.

Methodology

The proposed bootstrapping approach focuses on three core components:

  1. ASR (Speech Recognition): Acoustic models are borrowed from existing applications. Language models are constructed by combining data from previous apps, human-human dialog corpora (e.g., Switchboard), and text scraped from the target domain's web pages.
  2. SLU (Language Understanding): User intents are organized into a "Call-type Library" categorized as Generic (common across all apps, like "Hello" or "Operator"), Re-usable (industry-specific, like "Request Balance"), or Specific (unique to the business).
  3. DM (Dialog Manager): The system utilizes Augmented Transition Networks (ATNs) and reusable sub-dialog templates for common tasks like collecting phone numbers or addresses.
Architecture of the Spoken Dialog System framework.

The framework was evaluated by bootstrapping a pharmaceutical application using data from telecommunications (App 1) and medical insurance (App 2) domains, supplemented by web-scraped content.

Key Insights and Results

A critical finding of this study is that human-machine dialog data from disparate domains is often more effective for ASR than human-human data from the same domain. This is attributed to the distinct way people interact with machines—using shorter, clearer phrases with fewer disfluencies compared to natural human conversation.

ASR Word Accuracy vs. Run Time for various data combinations.

The bootstrapped system, despite having no exposure to pharmaceutical-specific training data, achieved notable performance:

  • ASR Performance: Reached approximately 67% word accuracy through the strategic combination of out-of-domain and web data. As shown in the performance curves, combining multiple data sources significantly outperforms generic human-human corpora like Switchboard.
  • SLU Performance: The model achieved 79% classification accuracy with 70% coverage of total utterances. Notably, handling "Generic" intents immediately addressed about 10% of user interactions.
  • Routing Success: In simulated dialogs, the system correctly routed 62% of callers, even with limited coverage of destination categories.

Conclusion

Reusing call-type libraries not only accelerates development but also enforces consistency across projects. While the system currently requires expert oversight for brand-specific "voice" alignment, the results suggest that performance will scale as the library of reusable applications grows. Future research will focus on automated "tuning" of these bootstrapped models using live data collected during early deployment.

Summary updated

BibTeX
@inproceedings{fabbrizio2004bootstrapping,
  author = {Giuseppe Di Fabbrizio and Gökhan Tür and Dilek Hakkani-Tür},
  title = {{Bootstrapping Spoken Dialog Systems with Data Reuse}},
  booktitle = {SIGDIAL Workshop},
  year = {2004},
  url = {https://openreview.net/forum?id=zUZ2YVPbbSz},
}
Figure