Abstract
Dysarthria, a motor speech disorder often resulting from neurological damage such as stroke, significantly impacts a patient's quality of life. While mechanical speech impairments are well-documented, the emotional dimensions of pathological speech remain under-researched, particularly in Mandarin Chinese. This paper introduces THE-POSSD, the first Chinese multimodal emotional pathological speech database. By integrating audio, video, glottal signals, and brain activity (fNIRS), the study demonstrates that motor dysarthria not only impairs speech intelligibility but also significantly degrades emotional expression, correlating strongly with psychological distress.
Introduction
Patients with dysarthria frequently experience affective disorders like depression and anxiety. However, most existing speech databases are "emotion-neutral" and limited to single modalities. THE-POSSD (Multimodal Emotional Database of Post-Stroke Speech Dysfunction Patients) addresses this gap by providing a comprehensive dataset to study the intersection of motor impairment, emotional expression, and psychological health in Mandarin speakers.
Methods and Data Collection
The researchers constructed THE-POSSD using a rigorous multimodal pipeline involving 68 participants (39 post-stroke patients and 29 healthy controls).
Multimodal Capture
Data collection included:
- Acoustic & Glottal Signals: High-quality audio paired with Electroglottograph (EGG) data.
- Physiological & Brain Activity: 22-channel functional near-infrared spectroscopy (fNIRS) for brain oxygenation and pulse oximetry for heart rate.
- Visual Data: High-definition video for facial expression analysis.
Emotion Induction
To elicit genuine emotions (Happy, Sad, Angry, Neutral), the study utilized video clips and "Stanislavsky’s emotional memory approach," where subjects recalled personal experiences to trigger specific affective states.
Key Insights: The Emotional Toll of Dysarthria
Psychological Correlation
Using the Symptom Checklist-90 (SCL-90), the study found that psychological factors—including depression, anxiety, and interpersonal sensitivity—are significantly higher in dysarthria patients compared to healthy controls. These psychological symptoms scale directly with the severity of the motor impairment.
Emotional "Flattening" and Confusion
The study confirms that as physical speech ability declines, the ability to convey emotion through voice degrades, leading to a "flattening" of expression. Interestingly, in the Valence-Arousal (VA) spatial distribution, "happy" expressions in patients often crossed into the "angry" quadrant. This occurs because patients frequently use a strained or "excited" voice to compensate for motor limitations, which can confuse both human listeners and automated algorithms.
Evaluation and Results
Human vs. Machine Recognition
- Subjective Accuracy: Human annotators achieved high emotion discrimination (above 87.5%), with neutral emotions being the easiest to identify.
- Speech Intelligibility: While healthy controls were rated at 93.8% intelligibility, this dropped to 67.3% for severe dysarthria patients.
- Automated Recognition: Machine learning models (SVM and Random Forest) achieved 78% accuracy for controls but only 60% for patients using audio data. Glottal data proved even more challenging for emotion classification, though it remained highly effective for classifying disease severity (up to 87% accuracy).
Conclusion and Future Work
THE-POSSD provides a unique resource for the study of pathological speech. While acoustic data remains superior for emotion recognition, glottal and physiological data offer critical insights into the physical impact of the disease. Future research will focus on the fusion of video and fNIRS data to create a more robust, multidimensional model of pathological speech and its emotional components.
Summary updated