Introduction
In clinical settings, speech patterns—such as pauses, turn-taking behavior, and speech rate—serve as vital "digital biomarkers" for neurodegenerative conditions like Huntington’s Disease (HD). However, manual annotation of long clinical interviews is labor-intensive and requires specialized expertise. This study investigates automated "who spoke when" (diarization) techniques, specifically comparing general speaker identification against role-specific recognition in a psychologist-patient context.
Methods
The researchers evaluated two primary architectural approaches to automate diarization in clinical interviews:
- Speaker Role Recognition (SRR): An end-to-end neural network that processes raw audio via SincNet filters and Bi-LSTM layers. It is trained to directly classify segments as "Psychologist" or "Patient," capturing role-specific prosodic and linguistic traits rather than individual voice identities.
- Speaker Enrollment Protocol (SEP): A traditional multi-stage pipeline involving Voice Activity Detection (VAD), Speaker Change Detection (SCD), and speaker matching using X-vector embeddings against a pre-enrolled voice template of the speaker.
To ensure generalizability, a "meta-split" strategy was used, testing models on entirely unseen speakers to prevent the networks from simply memorizing individual voices from the training set.
Datasets
The study utilized a specialized dataset of 94 clinical interviews from Henri-Mondor Hospital, France. The participants were categorized into three groups: Healthy Controls (22), Pre-manifest gene carriers (11), and Manifest HD patients (33). Models were pre-trained on large-scale public datasets like DIHARD and VoxCeleb2 before being fine-tuned on this specialized clinical data to account for hospital acoustics and dysarthric speech patterns.
Key Results
The experimental results highlighted the superiority of end-to-end modeling over traditional pipelines in clinical environments.
- Performance Lead: The end-to-end SRR model achieved the lowest Identification Error Rate (IER) of 19.5%. In contrast, the best-performing SEP pipeline (utilizing fine-tuned embeddings and retrained VAD/SCD) reached an IER of 23.6%.
- Error Propagation: Multi-stage pipelines suffered from "error accumulation," where inaccuracies in early stages (like VAD) negatively impacted subsequent speaker identification. The end-to-end approach avoids this by optimizing for the final task directly.
- Fine-tuning Necessity: Models trained only on general datasets (e.g., YouTube or news) performed poorly. Domain-specific fine-tuning was essential to handle the unique acoustic environment of a hospital and the specific speech patterns of patients with neurodegenerative disorders.
Clinical Implications: The "Silence Ratio"
Beyond technical accuracy, the study validated the clinical utility of automated diarization. The system successfully extracted the Ratio of Silence, a metric that typically increases as Huntington’s Disease progresses. The automatically derived silence ratio showed a significant correlation with ground truth annotations (r = 0.579, p = 0.009), suggesting it can serve as a reliable automated biomarker for monitoring disease state and progression.
Conclusion
While the models excel at providing global interview summaries, precision in fine-grained turn-taking remains a challenge, often due to background noise in hospital environments. Future work aims to refine these temporal detections and explore direct prediction of clinical scores, such as the Total Motor Score, to enable non-invasive, home-based patient monitoring.
Summary updated