TL;DR
This paper introduces a novel approach to automatically detect stuttering events in children's speech by adapting Automatic Speech Recognition (ASR) systems. By using "task-oriented lattices" and a separate "prolongation detector," the system significantly improves the identification of various stuttering types, paving the way for more objective and efficient diagnosis.
What Problem Does This Paper Address?
Stuttering is a common speech disorder that typically emerges in early childhood. Accurate diagnosis is crucial for timely intervention, as delays can lead to long-term social and mental difficulties. Clinicians currently diagnose stuttering by manually counting and classifying stuttering events during reading tasks. This process is time-consuming, subjective, and requires significant expertise. While Automatic Speech Recognition (ASR) systems are widely used in speech pathology, they traditionally aim to produce clean, fluent transcriptions, often by discarding disfluencies like stuttering. This paper addresses the challenge of adapting ASR technology to accurately detect and transcribe these stuttering events in children's speech, which is inherently more difficult for ASR systems due to factors like smaller vocal tracts and variable speech rates.
Methods
The authors propose a two-pronged approach:
- Lightly-Supervised ASR with Task-Oriented Lattices: Instead of relying on standard ASR models that filter out disfluencies, this method uses a "lightly-supervised" approach. The core idea is to leverage the original reading prompt (OP) as a rough, initial transcript. This OP is then used to generate "task-oriented lattices." These lattices are essentially ASR hypotheses that are biased to include potential stuttering events (like repetitions and revisions) by adding specific arcs to the decoding graph. This allows the ASR to consider these disfluencies during transcription, rather than discarding them.
- Prolongation Detection System: Prolongation events (e.g., stretching out sounds like "mmmommy") are particularly challenging for standard ASR. To address this, a separate "prolongation detector" is employed. This system uses an autocorrelation algorithm to measure the similarity between successive speech frames, identifying segments that exhibit characteristics of prolongations. This detector acts as a correction layer, adding information about detected prolongations to the ASR's output.
The integrated system first extracts MFCC features from the speech signal. These features are then processed in parallel by the ASR and the prolongation detector. Finally, the outputs are aligned to produce a detailed verbatim transcript that includes detected stuttering events.
Datasets
- UCLASS stuttering corpus (Release Two): Used for training the ASR model. This corpus contains 107 recordings of children reading.
- PF-Star corpus: 7 hours of children's read speech from this corpus were added to the training set to improve the ASR model.
- UCLASS Release One: 31 recordings from this corpus, which already had transcriptions, were used to establish transcription conventions.
The authors transcribed 48 speech samples from the UCLASS corpus for their research, dividing them into training, testing, and development sets.
Key Insights and Ideas
- ASR can be repurposed for disfluency detection: By modifying the ASR decoding process (using task-oriented lattices), it's possible to make ASR systems sensitive to stuttering events, rather than just fluent speech.
- Task-specific lattices are crucial: Standard ASR models are designed to remove disfluencies. Task-oriented lattices, which are specifically designed to allow for repetitions and revisions, are key to preserving these events in the output.
- Prolongations require a separate approach: Standard ASR struggles with the temporal characteristics of prolongations. A dedicated signal processing-based detector is necessary to effectively identify these events.
- Children's speech is a unique challenge: The paper acknowledges that children's speech presents inherent difficulties for ASR, and detecting stuttering adds another layer of complexity.
Key Results
- The lightly-supervised ASR approach using task-oriented lattices improved the Word Error Rate (WER) by 27.8% relative to a baseline that used standard word lattices.
- This approach preserved 63% of stuttering events, including sound, word, part-word repetitions, and revisions.
- The integrated system, which combined the ASR with the prolongation detector, increased the percentage of preserved stuttering events to 70%.
- The prolongation detector achieved a 0% miss rate on the test set, though it had a 5.3% false alarm rate.
- The overall system achieved a 6.25% false alarm rate for all stuttering events after integration.
Limitations and Future Work
- False alarms in prolongation detection: Artefact noises like background noise and heavy breathing were sometimes misidentified as prolongations. The authors suggest that a better silence remover could minimize these false alarms.
- Limited stuttering types in evaluation: While the system could detect part-word and phrase repetitions, these types were not present in the evaluation data.
- Future work: The authors plan to:
- Minimize false alarms in the prolongation detector by applying a better silence remover.
- Add more features to the task-oriented lattices to recognize interjection events.
Summary updated