Topic-Aware Dialogue Speech Recognition with Transfer Learning

Yuanfeng Song, Di Jiang, Xueyang Wu, Qian Xu, Raymond Chi-Wing Wong, Qiang YangInterspeech 2019 · OpenReview

TL;DR

This paper introduces a new topic model (DSTM) specifically designed to capture the nuances of dialogue speech, leading to improved Automatic Speech Recognition (ASR). To overcome the computational cost of training DSTM, they propose a novel transfer learning approach that leverages knowledge from existing topic models, significantly reducing training time without sacrificing performance.

What Problem Does This Paper Address?

Traditional Automatic Speech Recognition (ASR) systems often treat speech as a sequence of words without deeply considering the underlying context or topic of conversation. This is particularly problematic for dialogue speech (like chitchat, meetings, or customer service), where topic shifts and word repetition (burstiness) are common. Ignoring this topic information limits the accuracy of ASR systems. Existing topic models, while useful for general text, are not optimized for the unique characteristics of dialogue, such as utterance boundaries and the tendency for related words to appear together.

Methods

The core of this paper is the Dialogue Speech Topic Model (DSTM). DSTM is designed to address the limitations of previous topic models by:

  • Modeling Utterance Boundaries: It assumes that words within a single utterance are likely to belong to the same topic, capturing a key structural element of dialogue.
  • Capturing Word Burstiness: It models the phenomenon where once a word related to a topic appears, similar words are more likely to follow.

However, training DSTM from scratch is computationally expensive. To mitigate this, the authors propose a transfer learning mechanism. This approach leverages a pre-trained, computationally cheaper topic model like Latent Dirichlet Allocation (LDA). Instead of training DSTM from zero, they transfer key parameters (specifically, the prior distribution of word distributions for topics) from the LDA model to initialize DSTM. This "inductive transfer learning" significantly reduces the training time and memory requirements for DSTM while retaining its ability to accurately infer dialogue-specific topics.

Datasets

The experiments were conducted using approximately 1000 hours of dialogue speech collected from real-life customer service interactions in Mandarin Chinese. The data was split into training (80%), development (10%), and testing (10%) sets.

Key Insights and Ideas

  • Dialogue has unique linguistic structures: Utterance boundaries and word burstiness are not just quirks but crucial features that can be exploited for better speech recognition.
  • Topic modeling can significantly boost ASR: By understanding the topic of a dialogue, ASR systems can make more informed predictions, leading to fewer errors.
  • Transfer learning is a game-changer for complex models: The proposed transfer learning method makes sophisticated topic models like DSTM practical for real-world applications by drastically cutting down on training costs.
  • The proposed DSTM is superior to general topic models for dialogue: Experiments show DSTM-based adaptations outperform those using standard LDA or other language model adaptation techniques.

Key Results

  • Perplexity: Language models adapted using DSTM (both trained from scratch and with transfer learning) achieved significantly lower perplexity scores compared to baseline n-gram models and n-gram models adapted with LDA or RNNLM. For example, n-gram + DSTM-S achieved a perplexity of 46.62, while the baseline n-gram was 59.10.
  • Word Error Rate (WER): DSTM-based adaptations also led to substantial reductions in WER. n-gram + DSTM-S achieved a 1.48% improvement over the baseline, and importantly, n-gram + DSTM-TL and n-gram + DSTM-TL-E (transfer learning versions) showed comparable performance to DSTM-S, indicating minimal performance degradation from transfer learning.
  • Efficiency: Training DSTM from scratch (DSTM-S) was very time-consuming (e.g., 4512.19 seconds for 50 topics). However, the transfer learning approach (DSTM-TL and DSTM-TL-E) drastically reduced training time to near LDA levels (e.g., 406.72 seconds for 50 topics), with inference times being even faster (around 1 second).

Limitations and Future Work

  • Limitations: The paper acknowledges that while transfer learning reduces training time, there might be a slight performance degradation compared to training DSTM from scratch. The authors state that this degradation is "mild."
  • Future Work: The authors plan to explore the intrinsic structures of dialogue speech further and investigate new topic models for language model adaptation. They also aim to apply these techniques to other dialogue-related tasks.

Summary updated

Abstract

Dialogue speech widely exists in scenarios such as chitchat, meeting and customer service. General-purpose speech recognition systems usually neglect the topic information in the context of dialogue speech, which has great potential for improving the performance of speech recognition. In this paper, we propose a transfer learning mechanism to conduct topic-aware recognition for dialogue speech. We first propose a new probabilistic topic model named Dialogue Speech Topic Model (DSTM) that is specialized for modeling the context of dialogue speech. We further propose a novel transfer learning mechanism for DSTM to significantly reduce its training cost while preserving its effectiveness for accurate topic inference. The experiment results demonstrate that proposed techniques in language model adaptation effectively improve the performance of the state-of-the-art Automatic Speech Recognition (ASR) system.
BibTeX
@inproceedings{song2019topic,
  author = {Yuanfeng Song and Di Jiang and Xueyang Wu and Qian Xu and Raymond Chi-Wing Wong and Qiang Yang},
  title = {{Topic-Aware Dialogue Speech Recognition with Transfer Learning}},
  booktitle = {Interspeech},
  year = {2019},
  url = {https://openreview.net/forum?id=EwIcGop0jH},
}
Figure