Introduction
Pathological speech, resulting from physical trauma such as head and neck tumors or neurological disorders, presents significant challenges for automated processing. The high degree of variability—stemming from tumor location, surgical intervention types, and native dialects—makes objective assessment difficult. Currently, clinical intelligibility assessment relies on subjective expert judgment, which is resource-intensive and prone to inconsistency. This research proposes an automated framework to classify speech as "Intelligible" (I) or "Not Intelligible" (NI) using a fusion of specialized descriptors.
Methodology
The proposed system decomposes speech analysis into three distinct high-level subsystems designed to capture different layers of acoustic information:
- Multiple Language Phoneme Probability (MLPP): This approach utilizes phoneme recognizers from Czech, Hungarian, and Russian to analyze Dutch speech. By measuring the likelihood of sounds belonging to these foreign languages, the system captures acoustic deviations characteristic of pathology without requiring extensive native-language training data.
- Prosodic and Intonational Features: This subsystem analyzes the "melody" of speech, including pitch variance, duration of voiced segments, and pitch trajectory smoothness. These features are critical as patients often exhibit disrupted rhythm and intonation.
- Voice Quality and Pronunciation: This focuses on vocal "texture" (jitter, shimmer, harmonics-to-noise ratio) and traditional markers like formant frequencies and pause ratios.
Joint Classification Innovation
A key innovation of this work is the "Joint Classification" technique. Rather than treating utterances as independent data points, the researchers employed Agglomerative Hierarchical Clustering (AHC) to group acoustically similar utterances. A majority voting rule was then applied within these clusters to ensure label consistency. For instance, if a high percentage of utterances in a cluster were predicted as "Not Intelligible," the system would adjust outliers to match the cluster majority, effectively smoothing classification errors based on acoustic similarity.
Experimental Setup
The study utilized the NKI CCRT Speech Corpus, provided for the Interspeech 2012 Speaker Trait Challenge. The dataset consists of sentence-level Dutch speech from patients treated for head and neck tumors. The system's performance was evaluated against standard challenge baselines using Support Vector Machines (SVM) and Random Forest (RF) classifiers.
Results and Discussion
Individual subsystem analysis revealed that prosodic and intonational features were the most effective single descriptors, achieving 67.1% accuracy, outperforming the individual voice quality and pronunciation subsystem.
As shown in the results, the Joint Classification approach—which fused prosodic features with selected baseline functionals and applied cluster-based smoothing—achieved the highest performance: 79.9% on the development set and 76.8% on the test set. This significantly outperformed the best baseline system (Random Forest at 68.9%). Interestingly, complex Bayesian Network fusion did not perform as well as the simpler joint classification, likely because the dataset was too small to properly train the complex conditional probability tables required for Bayesian inference.
Conclusion
The research demonstrates that enforcing acoustic consistency through clustering significantly improves intelligibility classification in pathological speech. The success of cross-lingual phoneme probabilities and prosodic features highlights the value of high-level descriptors in capturing the nuances of speech disorders. Future work will explore integrating voice clustering directly into the training phase and investigating the model's performance on spontaneous, natural conversation.
Summary updated