TL;DR
This paper systematically investigates how different acoustic features and acoustic models impact the recognition of dysarthric speech, a challenging disorder characterized by significant speech variability. By exploring combinations of spectral features (like MFCCs and FBANKs) with pitch information and evaluating them across various models, the authors found that specific feature-model pairings significantly improve recognition accuracy, particularly for sentence-level tasks. Their optimized F-TDNN model achieved notable performance gains, demonstrating the importance of tailored approaches for this specialized ASR domain.
What Problem Does This Paper Address?
Dysarthric speech, caused by neurological impairments affecting motor control of speech, presents a major hurdle for Automatic Speech Recognition (ASR) systems. The core challenge lies in the extreme acoustic variability and lack of clarity inherent in dysarthric speech, which deviates significantly from typical speech patterns. While ASR technology has advanced rapidly for unimpaired speech, achieving comparable performance for dysarthric speakers remains a critical research area. This paper aims to bridge this gap by systematically studying how different acoustic features and acoustic models interact to improve the recognition of dysarthric speech, addressing the need for more robust and accessible ASR solutions for individuals with this condition.
Methods
The researchers conducted a comprehensive experimental study comparing various acoustic features and acoustic models for dysarthric speech recognition.
- Acoustic Features: They evaluated standard spectral features like FBANKs, MFCCs, and PLPCCs. Crucially, they also investigated the impact of incorporating pitch features, hypothesizing that this would help capture dysarthric speech's characteristic pitch variations. Combinations of these features were also explored.
- Acoustic Models: Five different acoustic models were employed:
- HMM-GMM (Hidden Markov Model with Gaussian Mixture Model)
- SGMM (Sub-space Gaussian Mixture Model)
- DNN (Deep Neural Network)
- TDNN-LSTM (Time Delay Neural Network with Long Short-Term Memory)
- F-TDNN (Factorized Time Delay Neural Network), which was the focus for optimization.
- Optimization: For the F-TDNN model, a key novel aspect was the systematic investigation of the number of overlapping frames between consecutive training chunks. This parameter is crucial for managing contextual information and mitigating speech variability in chain-based models. They also employed speed perturbation data augmentation to address the limited dataset size and speech tempo variations.
- Training and Evaluation: A leave-one-out cross-validation approach was used, with 14 speakers for training and 1 for evaluation. Two language models (unigram for isolated words, bigram for sentences) were utilized.
Datasets
The primary dataset used in this study is the TORGO Dysarthric Speech Database.
- TORGO Database: This is a publicly available resource specifically for dysarthric speech research. It contains approximately 15 hours of speech data from 8 dysarthric speakers (contributing ~6 hours) and 7 control speakers (contributing ~9 hours). The database includes 16,394 utterances, with a mix of single-word and sentence utterances. The paper also mentions using control speakers from this database for comparative analysis.
Key Insights and Ideas
- Pitch Features Matter: The inclusion of pitch features significantly improved recognition performance, especially for sentence recognition tasks involving dysarthric speech. This highlights that capturing prosodic variations is vital for understanding dysarthric speech.
- Feature-Model Synergy: There isn't a one-size-fits-all best feature or model. The optimal choice depends heavily on the specific acoustic model being used. For instance, HMM-GMM favored PLPCCs, while TDNN-LSTM excelled with MFCCs.
- F-TDNN Optimization is Key: The F-TDNN model, when optimized with the right parameters (specifically, overlapping frames), shows substantial promise. The systematic exploration of overlapping frames revealed that 20 frames was optimal for dysarthric speech recognition.
- Data Augmentation Pays Off: Speed perturbation, a form of data augmentation, helped improve performance, underscoring its utility for limited dysarthric speech datasets.
- Individual Variability is High: The speaker-specific analysis (Table III) clearly shows significant differences in recognition performance across individual dysarthric speakers, reinforcing the challenge of intra-speaker variability.
Key Results
- F-TDNN Improvements: The optimized F-TDNN model achieved a 4.65% relative improvement in isolated word recognition and a 4.63% relative improvement in sentence recognition for dysarthric speech compared to previous research [10].
- Best Feature Combinations:
- For isolated word recognition with F-TDNN, FBANKs+MFCCs+Pitch yielded the best result with a Word Error Rate (WER) of 41.0%.
- For sentence recognition with F-TDNN, MFCCs alone achieved the best performance with a WER of 24.7%.
- Model Performance:
- In general ASR tasks (not F-TDNN specific), TDNN-LSTM achieved the lowest WER (44.2%) for isolated word recognition, while DNN performed best (35.7% WER) for sentence recognition.
- Control Speaker Performance: The improvements and feature selections also showed benefits for control speakers, indicating the generalizability of the findings.
Limitations and Future Work
- Limitations:
- The study acknowledges the inherent challenge of limited dysarthric speech data, which necessitates techniques like data augmentation.
- While the F-TDNN model was optimized, the paper focuses on specific feature sets and model configurations, implying that further exploration might yield even better results.
- The severity labels for speakers were derived from isolated word recognition WERs, which might not fully capture the nuances of sentence-level performance.
- Future Work:
- The authors suggest further enhancing DyASR by addressing speech variations, employing speech enhancement methods, and exploring more advanced data augmentation techniques.
- They mention preliminary investigations using a Wav2Vec2-based end-to-end ASR system, indicating a future direction towards leveraging self-supervised learning models.
Summary updated