Towards an ASR-free objective analysis of pathological speech

Catherine Middag, Yvan Saeys, Jean-Pierre MartensInterspeech 2010

TL;DR

This paper introduces a novel, ASR-free method for objectively analyzing pathological speech by extracting phonological features directly from raw audio. This approach bypasses the need for speech recognition, making it more robust to speech errors and potentially offering deeper insights into articulatory problems than traditional intelligibility measures.

What Problem Does This Paper Address?

Assessing speech intelligibility, especially for individuals with speech disorders, is crucial for monitoring therapy progress and understanding communication effectiveness. Traditionally, this relies on perceptual tests with human listeners, which are time-consuming and subjective. While Automatic Speech Recognition (ASR) systems have shown promise in automating this, they struggle with "out-of-vocabulary" words and errors common in pathological speech. This paper tackles the challenge of developing an objective speech analysis method that is independent of ASR, aiming to provide a more reliable and insightful measure of speech pathology.

Methods

The core of the proposed method is an "ASR-free" approach that analyzes running speech. It involves three main stages:

  1. Acoustic Analysis (Front-End): Standard Mel-Frequency Cepstral Coefficients (MFCCs) are extracted from the audio signal.
  2. Phonological Feature Extraction: Artificial Neural Networks (ANNs) are trained to detect 14 distinct phonological features (e.g., voicing, place of articulation, nasality) directly from the acoustic features. Crucially, this stage does not require knowing what the speaker was supposed to say. The ANNs output confidence scores for the presence/absence and relevance/irrelevance of these features for each time frame.
  3. Phonological Characterization: Statistical analysis is performed on the phonological feature confidence scores over the entire utterance. This analysis generates a compact feature vector for each speaker, capturing patterns and fluctuations in their phonological production. This vector is then used for prediction tasks.

Datasets

The study utilizes a portion of the Dutch Corpus of Pathological and Normal Speech (COPAS). This corpus includes recordings from 318 Flemish speakers, both pathological and control. For 122 of these speakers, read text passages were recorded, which were used for the experiments. Perceptual Phoneme Intelligibility (PI) scores derived from the Dutch Intelligibility Assessment (DIA) were available for all speakers, but Running Speech Intelligibility (RSI) scores were not directly available for the read passages.

Key Insights and Ideas

  • ASR Independence is Key: The paper demonstrates that valuable information about speech pathology can be extracted without relying on ASR, overcoming limitations of existing automated methods.
  • Phonological Features as Articulatory Proxies: The extracted phonological features are hypothesized to be closely related to articulatory dimensions, offering a more direct window into the speaker's motor control problems.
  • Running Speech is More Relevant: Analyzing running speech, rather than isolated words, is presented as a more ecologically valid measure of communication efficiency.
  • Statistical Analysis Reveals Pathology: Fluctuations and patterns in phonological feature production over time, even without knowing the intended speech, can reveal articulatory deficiencies.

Key Results

  • PI Prediction: The ASR-free method achieved comparable performance to ASR-based methods in predicting Phoneme Intelligibility (PI), with RMSE values around 9.7-9.8. This suggests the extracted features are a good proxy for intelligibility.
  • Pathology Classification: While ASR-based features slightly outperformed ASR-free features in classifying pathological vs. normal speakers (error rates around 8.2-13.0% for SVM), the ASR-free features still showed promise. A specific rule derived from ASR-free features correctly classified 58% of normal and 99% of pathological speakers, highlighting the importance of "relevancy" features.
  • Speaker Profiling: The ASR-free features enabled better discrimination between normal and hearing-impaired speakers (up to 92% accuracy) compared to ASR-based features (up to 87%). This was exemplified by identifying hypernasality in hearing-impaired speakers. For laryngectomees, features related to turbulence and voicing were found to be discriminating.

Limitations and Future Work

  • Limited Running Speech Data: The authors acknowledge that they had insufficient running speech recordings and corresponding RSI scores for comprehensive model training and evaluation.
  • Exploratory Nature: The current study is described as an "exploratory investigation" due to data limitations.
  • Future Work: The authors plan to collect more running speech recordings and RSI scores to develop more robust diagnosing systems. They also aim to improve speaker feature extraction and incorporate segmental (e.g., "trill") and supra-segmental (e.g., intonation) features for a more complete analysis. Further investigation into the specific articulatory dimensions encoded by the features is also planned.

Summary updated

BibTeX
@inproceedings{middag2010towards,
  author = {Catherine Middag and Yvan Saeys and Jean-Pierre Martens},
  title = {{Towards an ASR-free objective analysis of pathological speech}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure