TL;DR
This paper explores using telephone-quality speech recordings to assess cognitive function in the elderly. They demonstrate that speech characteristics can be reliably extracted from these recordings with minor adjustments to feature extraction algorithms, paving the way for cost-effective, continuous cognitive monitoring.
What Problem Does This Paper Address?
Assessing cognitive function, especially in the elderly, is a critical but challenging task. Traditional clinical assessments are time-consuming, expensive, and only provide a snapshot in time. Cognitive decline is a gradual process, and early detection is crucial for timely intervention. The authors propose that speech, a readily available and natural human behavior, can serve as a valuable, continuous indicator of cognitive health. However, the practicalities of remote monitoring via telephone present challenges due to lower audio quality, noise, and variable recording conditions compared to controlled lab environments. This paper aims to bridge this gap by investigating how to reliably extract cognitive indicators from telephone speech.
Methods
The core of this work lies in adapting existing speech feature extraction techniques for telephone-quality audio. The authors focus on temporal speech features, hypothesizing that cognitive impairment affects speech production timing, leading to longer pauses and altered utterance durations.
Key methodological contributions include:
- Speech/Non-speech Threshold Estimation: A robust method to differentiate speech from silence, breaths, and other non-speech artifacts. This involves full-wave rectification, energy calculation per frame, and establishing a threshold based on the lowest energy frames.
- Gradient Threshold for Breath Sound Detection: Recognizing that breaths can be mistaken for pauses, they exploit the difference in the amplitude envelope gradient between speech and breath sounds. A lower gradient is indicative of a breath sound.
- Reference Value for Gradient Threshold: To account for speaker-specific variations in breath sound characteristics, a "Reference Value" is introduced. This value normalizes the gradient threshold, making it more universally applicable.
- Breath Sound Detection Algorithm: This algorithm uses Hilbert transform and low-pass filtering to estimate the amplitude envelope. It then applies the gradient threshold and an amplitude-based condition (below 120% of the Speech/Non-Speech Threshold) to identify breath sounds. Segments shorter than 100ms are then classified as speech.
- Edge Detection Algorithm: To precisely demarcate breath sound boundaries, an edge detection algorithm is employed. This involves extending breath sound segments, calculating energy contours, and identifying local minimums to refine the segment edges.
- Feature Extraction: After segmenting speech and non-speech, seven features are extracted: number of pauses, mean pause duration, proportion of recording in silence, mean utterance duration, total recording time, total length of pauses, and total length of utterances.
- Adaptations for Telephone Speech: Two crucial modifications were made for telephone recordings:
- A seven-point running average filter was applied to smooth the signal and suppress "clicks" and "knocks" that could be misinterpreted.
- The Reference Value for the Gradient Threshold was re-estimated specifically for telephone recordings to account for the different noise characteristics.
Datasets
The study utilized recordings from 19 subjects who participated in a larger study on the elderly at St. James's Hospital, Dublin. These subjects underwent a battery of cognitive tests.
- Clinic Recordings: Subjects read a fixed passage of text (390 words) in a controlled environment. Recordings were made using a laptop with an external soundcard, sampled at 44.1 kHz, 16-bit resolution, and uncompressed.
- Telephone Recordings: The same subjects were later called at home, and cognitive interviews were recorded over the phone. These recordings were sampled at 8 kHz, 16-bit resolution, and used CCITT µ-LAW compression.
The paper compares features extracted from both clinic and telephone recordings for these 19 subjects.
Key Insights and Ideas
- Speech as a Continuous Cognitive Monitor: The fundamental idea that speech patterns can reflect cognitive health is reinforced. The ability to extract these patterns from everyday telephone conversations is a significant step towards unobtrusive, continuous monitoring.
- Robustness of Temporal Features: Despite the significant degradation in audio quality from clinic to telephone, the core temporal features (pause duration, utterance duration) remain largely stable. This suggests these features are robust indicators of cognitive function.
- The Importance of Noise and Artifact Handling: The paper highlights the necessity of specific pre-processing steps (filtering, re-estimation of thresholds) to handle the unique challenges of telephone audio, such as clicks, knocks, and variable noise levels.
- Potential for Automated Screening: The success in reliably extracting features from telephone speech opens the door for fully automated Interactive Voice Response (IVR) systems, which could significantly reduce the cost and increase the scale of cognitive screening.
Key Results
The feature extraction algorithm demonstrated strong performance on telephone-quality speech:
- Overall Accuracy: 93.2%
- Sensitivity: 97.3%
- Specificity: 89.5%
- Positive Predictivity: 91.1%
- Negative Predictivity: 97.1%
Comparisons of average feature values between clinic and telephone recordings for 19 subjects showed differences of less than 15% for most features. The "Total Length of Pauses" showed a slightly larger difference (around 19%), but these differences were not statistically significant, supporting the hypothesis that telephone speech can be used for cognitive assessment.
Limitations and Future Work
- Limitations:
- The study used a relatively small cohort of 19 subjects for the direct comparison of clinic vs. telephone recordings.
- The "Reference Value" for the Gradient Threshold still required re-estimation for telephone recordings, indicating a need for further automation.
- The recordings were made by trainee psychologists, not fully automated.
- Future Work:
- Investigate changes in extracted features over time in a longitudinal study, with recordings taken every two months.
- Develop a fully automated IVR system for cognitive assessment and monitoring.
- Develop automatic Reference Value estimation to eliminate the need for manual re-estimation for different recording systems.
- Explore the use of speech for monitoring cognitive function in larger cohorts.
Summary updated