Topic and style-adapted language modeling for Thai broadcast news ASR

Markpong Jongtaveesataporn, Sadaoki FuruiInterspeech 2010

TL;DR

This paper tackles the challenge of building accurate speech recognition systems for Thai broadcast news, a language with limited training data. They propose a novel approach to adapt language models by leveraging newspaper text and classifying broadcast news text into "spoken" and "written" styles, significantly improving recognition accuracy.

What Problem Does This Paper Address?

Developing robust Automatic Speech Recognition (ASR) systems, especially for languages like Thai, is heavily reliant on large amounts of transcribed text data to train effective language models (LMs). For Thai broadcast news (BN), this data is scarce and costly to produce. While newspaper (NP) text is more readily available and shares some similarities with BN, it's not a perfect substitute. Furthermore, even within BN, there are distinct speaking styles (e.g., formal news reading vs. informal introductions/transitions) that can impact LM performance. This paper aims to bridge this data gap and account for stylistic variations by adapting LMs using both NP text and stylistically classified BN text.

Methods

The core of this work lies in language model adaptation techniques. They explore several strategies for creating specialized LMs and combining them:

  • Text Source Adaptation (Model Type I): Training separate LMs for BN and NP text and interpolating them.
  • Speaking Style Adaptation (Model Type II): Developing a rule-based classifier to distinguish between "spoken" and "written" styles within the text. This classifier identifies specific "spoken style ending words" (SSEWs) that are characteristic of informal speech. LMs are then trained for each style.
  • Combined Topic and Style Adaptation (Model Types III-VII): This is where the novelty truly shines. They combine adaptation based on:
  • Source and Style (Model Type III): Adapting to BN/NP and spoken/written styles simultaneously.
  • Topic (Model Type IV): Clustering text by topic using TF-IDF and cosine similarity, then training LMs for each topic.
  • Source, Topic, and Style (Model Types V, VI, VII): These models represent increasingly granular adaptation, combining source, topic, and style information to create highly specialized LMs.

All specialized LMs are then combined using linear interpolation, with weights optimized using an EM algorithm on recognition hypotheses.

Datasets

  • Training Data:
    • Thai Broadcast News (BN) Corpus: Limited in size (around 1.8 million pseudo-morphemes).
  • Newspaper (NP) Text Corpus: Significantly larger (around 139 million pseudo-morphemes), covering news from 2003-2007.
  • Acoustic Training Data: 40.3 hours of transcribed speech from 68 male and 68 female speakers, used for training gender-dependent acoustic models.
  • Evaluation Data: A BN test set consisting of 1033 clean speech utterances (626 male, 407 female).

Key Insights and Ideas

  • The Power of Style Classification: A simple rule-based approach to classify text into "spoken" and "written" styles proved effective. This highlights that even subtle stylistic differences can be leveraged for LM adaptation.
  • Newspaper Text as a Valuable Supplement: NP text can effectively alleviate the sparseness of BN corpora, particularly for modeling "written" style speech.
  • BN Spoken Style is Crucial: The paper strongly emphasizes that spoken style text from the BN corpus is essential for building a reliable BN ASR system. This suggests prioritizing the transcription of spoken segments in future corpus development.
  • Unsupervised Adaptation Challenges: While supervised adaptation showed significant improvements, unsupervised adaptation (using recognition hypotheses) led to performance degradation. This is attributed to the difficulty in accurately estimating LM weights when hypotheses contain errors, especially for more complex adaptation schemes.
  • Model Type III's Practicality: Despite more complex models showing better results in supervised settings, Model Type III (adapting to source and style) offered a good balance of performance improvement and simpler LM training, making it a practical choice.

Key Results

  • Significant Reduction in Perplexity (PP) and Phoneme Error Rate (PER): All proposed adaptation methods, when applied in a supervised manner, successfully reduced PP and PER compared to a baseline LM trained on combined BN and NP text.
  • Best Supervised Performance: Model Type VII (adapting to source, topic, and style) achieved the best results in supervised adaptation, with a PP of 99.1 and PER of 15.9%.
  • Best Unsupervised Performance: Model Type V (adapting to source and topic) yielded the best results in unsupervised adaptation, with a PP of 127.1 and PER of 18.0%. This model outperformed Model Type VII in unsupervised settings, likely due to its simpler structure and robustness to errors in hypotheses.
  • Model Type III's Effectiveness: Model Type III (source and style adaptation) achieved a PER of 16.9% in supervised and 18.2% in unsupervised settings, demonstrating its practical utility.

Limitations and Future Work

  • Unsupervised Adaptation Robustness: The paper acknowledges that unsupervised adaptation is less robust than supervised adaptation, particularly for models with many parameters or complex clustering. Future work could focus on improving LM weight estimation in the presence of recognition errors.
  • Topic Clustering Refinement: The authors suggest that further research into topic clustering techniques is needed to improve the performance of topic-adapted models.
  • Exploring Other Interpolation Schemes: Investigating alternative methods for combining specialized LMs could lead to further performance gains.
  • Leveraging Web Corpora: The authors propose exploring web corpora like blogs and Twitter, which contain a large amount of spoken-style text, as a promising avenue for future data enhancement.
  • Prioritizing Spoken Content: For resource-deficient languages like Thai, the paper strongly recommends prioritizing the creation of BN corpora with a high proportion of spoken-style utterances.

Summary updated

BibTeX
@inproceedings{jongtaveesataporn2010topic,
  author = {Markpong Jongtaveesataporn and Sadaoki Furui},
  title = {{Topic and style-adapted language modeling for Thai broadcast news ASR}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure