TL;DR
This paper tackles the challenge of limited data for training AI models in clinical speech applications, particularly for conditions like dysarthria. They developed a novel method to artificially generate dysarthric speech from healthy speech using adversarial training, which significantly improves classification accuracy when used for data augmentation.
What Problem Does This Paper Address?
Training robust machine learning models, especially deep learning models, for speech applications typically requires vast amounts of labeled data. This is a major hurdle in clinical settings where collecting such data is difficult and expensive due to privacy concerns, patient access, and the sensitive nature of pathological speech. Consequently, clinical speech AI often relies on small datasets with limited speaker diversity. This paper focuses on dysarthria, a speech disorder common in conditions like Amyotrophic Lateral Sclerosis (ALS), where the speech can range from mildly slurred to severely unintelligible. The goal is to create a method to augment existing datasets with simulated dysarthric speech to improve the performance of AI models for clinical applications.
Methods
The core of the proposed method is a three-step voice conversion process to transform healthy speech into simulated dysarthric speech, specifically targeting ALS-related speech characteristics.
- Speaking Rate Modification: Healthy speech is first adjusted to match the slower speaking rate characteristic of ALS speech. This is achieved using the PSOLA technique, which modifies duration while preserving pitch.
- Spectral Feature Transformation: After rate modification, spectral features (Mel-cepstral coefficients - MCEPs, and band-aperiodicity parameters - BAPs) are extracted. These features are then transformed using a Deep Convolutional Generative Adversarial Network (DCGAN). The DCGAN is trained to make the spectral features of healthy speech resemble those of dysarthric speech, aiming to capture characteristics like imprecise articulation and hypernasality.
- Pitch Modification: Finally, the pitch contour is modified using a linear transformation. This step aims to reduce pitch variation, a common trait in ALS speech, by adjusting the healthy speech's pitch standard deviation to match that of the target ALS speakers.
The DCGAN architecture involves a generator that transforms healthy speech features and a discriminator that tries to distinguish between the generated features and actual ALS speech features.
Datasets
The study utilized a dysarthric speech dataset collected from 8 ALS speakers (with moderate to severe dysarthria) and 8 healthy speakers (4 male, 4 female) from Arizona State University. Each speaker read the same 80 short phrases. The first 70 phrases were used for training, and the remaining 10 for testing. For the pilot experiment, ataxic speech data was also used, comprising 16 ataxic speakers and 8 ALS speakers, each reading the same 80 phrases.
Key Insights and Ideas
- Adversarial Training for Clinical Speech: The paper demonstrates the potential of adversarial training, a technique more commonly seen in image generation, for synthesizing realistic pathological speech. This opens doors for creating synthetic data for under-resourced clinical domains.
- Multi-stage Transformation: The three-step approach (rate, spectral, pitch) is crucial. It breaks down the complex transformation into manageable components, each targeting a specific characteristic of dysarthric speech.
- Speaker-Independent Augmentation: By training the model to map groups of healthy speakers to groups of ALS speakers, the goal is to learn general dysarthric symptoms rather than specific speaker patterns, making the augmentation more broadly applicable.
- Subjective Validation is Key: The study emphasizes the importance of expert evaluation (Speech-Language Pathologists) to assess the perceptual realism of the generated speech, complementing objective acoustic measures.
Key Results
- Objective Evaluation: The transformed speech features were found to be significantly more similar to true ALS speech features than healthy speech features, as measured by Dp-divergence (0.552 for transformed vs. 0.669 for healthy vs. ALS). A Support Vector Machine (SVM) trained to classify ALS vs. healthy speech misclassified transformed healthy speech as ALS 37.5% of the time, compared to only 2.1% for original healthy speech.
- Subjective Evaluation: Experienced SLPs identified the transformed speech as dysarthric 65% of the time. When consensus was reached among clinicians, this figure rose to 76%. Importantly, the artifacts introduced by the transformation did not significantly hinder SLPs' ability to classify control healthy and ALS samples (98% accuracy).
- Data Augmentation Impact: In a pilot experiment to distinguish between ALS and ataxic speech, using the simulated dysarthric speech for data augmentation improved classification accuracy by approximately 10% compared to traditional augmentation methods (like adding noise).
Limitations and Future Work
- Perceptual Artifacts: The authors acknowledge that their transformation process introduces audible artifacts. While these didn't completely derail expert judgment, they are a limitation.
- Generalizability: The current work focuses on ALS speech. Further research is needed to explore its effectiveness for other types of dysarthria or speech disorders.
- Future Directions: The authors plan to explore other deep learning models, such as Recurrent Neural Networks (RNNs), which are known to be effective in voice transformation, to further reduce perceptual artifacts. They also aim to use the generated data to build larger-scale machine learning systems for clinical applications.
Summary updated