A comparative study of adaptive, automatic recognition of disordered speech

Heidi Christensen, Stuart Cunningham, Charles Fox, Phil Green, Thomas HainInterspeech 2012

TL;DR

This paper investigates how well standard automatic speech recognition (ASR) techniques, developed for typical speech, can be adapted to recognize dysarthric speech. They found that while initial performance is poor, adaptation techniques significantly improve accuracy, and the optimal system configuration varies greatly depending on the severity of the speaker's condition.

What Problem Does This Paper Address?

Speech-driven assistive technologies offer a promising avenue for individuals with physical disabilities who struggle with traditional interfaces. However, a common challenge is dysarthria, a speech disorder resulting from impaired motor control of the speech articulators. This often leads to speech that is difficult for standard ASR systems to understand. A major hurdle in developing effective ASR for dysarthric speech is the scarcity of suitable training data, as individuals with severe dysarthria may find it difficult or tiring to produce large amounts of speech. This paper aims to explore how well existing, sophisticated ASR techniques, refined on large datasets of typical speech, can be adapted to this challenging domain, and how to best leverage limited dysarthric speech data.

Methods

The researchers employed a variety of ASR system configurations, building upon established techniques from large-vocabulary continuous speech recognition (LVCSR). They started with:

  • Speaker Independent (SI) systems: Trained on a general corpus of typical speech (Mtg) or on a subset of the dysarthric data itself (SI1, SI2).
  • Speaker Dependent (SD) systems: Trained specifically on individual speaker data.

Crucially, they then applied adaptation techniques to these baseline systems:

  • Maximum Likelihood Linear Regression (MLLR) and Maximum A Posteriori (MAP) adaptation: These are standard methods in LVCSR to tune a general model to an individual speaker's voice using a small amount of adaptation data.
  • Domain Adaptation: They explored adapting models trained on typical speech (Mtg) to the dysarthric domain using the UAspeech data.
  • MAP-via-Domain Adaptation: A multi-stage approach where typical speech models were first adapted to the dysarthric domain and then further adapted to individual speakers.

Datasets

The primary dataset used in this study is UAspeech. This database contains synchronized audio and visual recordings of 15 dysarthric speakers (4 female, 11 male). The speakers repeated single words from various categories (digits, NATO alphabet, commands, common words, and uncommon words). The dataset includes intelligibility scores for each speaker, which is a key piece of meta-data. The authors also used a large corpus of typical speech data (over 170 hours of meeting data) for their baseline "Mtg" models. UAspeech is noted as being significantly smaller than typical speech databases, containing around 18 hours of speech after pre-processing, compared to the typical speech corpus used for training.

Key Insights and Ideas

  • Adaptation is Key: Even with a significant mismatch between typical and dysarthric speech, standard adaptation techniques (like MAP) can yield substantial improvements over baseline systems.
  • "One Size Fits All" Doesn't Apply: The optimal ASR system configuration is highly speaker-dependent. There isn't a single best approach that works for everyone.
  • Severity Matters: Speakers with more severe dysarthria show a stronger bias towards specific system configurations, indicating that the choice of system is more critical for them. Less severely impaired speakers tend to perform similarly across a wider range of systems.
  • Leveraging Typical Speech Data: Despite the initial poor performance of models trained on typical speech, adapting them to dysarthric data proves to be a viable strategy, demonstrating the power of transferring knowledge across domains.

Key Results

  • Baseline systems trained on typical speech (Mtg) performed poorly, with an average accuracy of 22.4%.
  • Speaker-independent (SI1, SI2) and speaker-dependent (SD) systems showed significantly better performance, with average accuracies around 50-51%.
  • MAP adaptation provided substantial improvements. For example, MAP adaptation of the typical speech models (mapMtg) increased average accuracy to 40.1%, an ~80% relative improvement. MAP adaptation of SI models (mapSI1, mapSI2) achieved average accuracies of 53.9% and 54.1%.
  • The best systems achieved average accuracies that were 34.5% higher than previously reported results on a subset of the UAspeech data.
  • The study visually demonstrated a correlation between dysarthria severity and system performance, with darker shades in a visualization indicating a stronger preference for certain systems among more severely impaired speakers.

Limitations and Future Work

  • Data Sparseness: While UAspeech is one of the largest available dysarthric speech databases, it is still an order of magnitude smaller than typical speech databases, posing a continuous challenge.
  • System Variability: The paper highlights that the "best" system varies significantly per speaker, and future work aims to develop systematic methods for identifying the optimal operating point for a given speaker.
  • Further Adaptation Techniques: The authors plan to investigate the effect of MLLR adaptation and discriminative training in future work.
  • Longitudinal Study: They intend to conduct a longitudinal study on the use of speech-driven assistive technology in home environments.

Summary updated

BibTeX
@inproceedings{christensen2012comparative,
  author = {Heidi Christensen and Stuart Cunningham and Charles Fox and Phil Green and Thomas Hain},
  title = {{A comparative study of adaptive, automatic recognition of disordered speech}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure