TL;DR
This paper explores using speech features to predict cortisol levels, a key biological stress marker, during a standardized stress test. They found that hand-crafted speech features, particularly from the COMPARE set, can predict cortisol levels with reasonable accuracy, suggesting speech analysis as a promising non-invasive method for stress monitoring.
What Problem Does This Paper Address?
Measuring stress in individuals often relies on invasive and time-consuming methods like collecting saliva or blood samples to quantify cortisol levels. While cortisol is a well-established indicator of the body's stress response, its sequential measurement is logistically challenging. The authors aim to bridge this gap by investigating whether speech, a readily accessible and non-invasive signal, can serve as a proxy for predicting these sequential cortisol measurements. This would enable more frequent and less intrusive stress monitoring.
Methods
The researchers utilized a multimodal dataset from the Trier Social Stress Test (TSST), a well-established protocol for inducing acute stress. They extracted three types of speech-based features:
- Hand-crafted features:
- COMPARE: A comprehensive set of 6373 acoustic features known for their effectiveness in paralinguistic tasks.
- EGEMAPS: A more concise set of 88 acoustic features.
- Deep features:
- DEEP SPECTRUM: Features extracted using pre-trained convolutional neural networks, representing a state-of-the-art approach.
These features were then used to train epsilon-support vector regression (SVR) models with a linear kernel. The models were trained to predict cortisol levels at various time points (from baseline to 60 minutes after the stress test). They evaluated performance using Spearman's correlation coefficient (ρ) and Root Mean Square Error (RMSE), also employing Cohen's d to measure effect size.
Datasets
The study utilized the FAU-TSST corpus, a novel multimodal dataset comprising audio recordings from 43 individuals undergoing the Trier Social Stress Test (TSST). This dataset includes sequential saliva samples for cortisol measurement and corresponding audio data. The corpus was split into speaker-independent training, validation, and test sets.
Key Insights and Ideas
- Speech as a Stress Biomarker: The core insight is that speech characteristics can indeed reflect physiological stress responses, as evidenced by their correlation with cortisol levels. This opens up new avenues for non-invasive stress monitoring.
- Delayed Cortisol Response and Speech: The study observed that the correlation between speech features and cortisol levels tends to improve around 10-20 minutes after the stressor (S3 and S4 time points). This aligns with the known delayed nature of cortisol release, suggesting that speech might capture the body's evolving stress response.
- Hand-crafted Features Shine: Surprisingly, traditional hand-crafted features (COMPARE and EGEMAPS) outperformed deep learning-based features (DEEP SPECTRUM) in this specific task. The authors speculate this might be due to the ambient noise in the recording environment affecting the spectral features extracted by deep models, or that hand-crafted features capture more relevant paralinguistic cues for stress.
- Scenario Differences: While differences were minimal, the arithmetic task within the TSST showed a slight tendency towards stronger correlations, possibly due to its more spontaneous and less prepared nature, leading to more pronounced stress indicators in speech.
Key Results
- The best prediction performance was achieved using COMPARE features at the S4 time point (20 minutes after the oral presentation), resulting in a Spearman's correlation coefficient (ρ) of 0.421 and a Root Mean Square Error (RMSE) of 0.245.
- This prediction also showed a significant Cohen's d of 0.883, indicating a large effect size between baseline and the 20-minute post-stress prediction.
- For the entire oral presentation scenario, COMPARE features achieved an RMSE of 0.244 at the S4 time point.
- EGEMAPS features achieved the highest correlation (ρ = 0.449) at the S3 time point (10 minutes after the oral presentation) in the arithmetic scenario.
- DEEP SPECTRUM features consistently yielded lower correlations and higher RMSEs across most scenarios.
Limitations and Future Work
- Feature Selection: The authors acknowledge that while hand-crafted features performed well, further investigation into specific features within these sets that are most predictive of cortisol would be beneficial.
- Deep Learning Performance: The underperformance of DEEP SPECTRUM features warrants further exploration, potentially through improved noise reduction techniques or different deep learning architectures.
- Multimodal Fusion: Given the availability of other modalities in the FAU-TSST corpus, future work could explore multimodal fusion approaches to enhance prediction accuracy.
- Word-Level Analysis: Investigating word usage and linguistic patterns during stressed speech could provide additional insights.
- Generalizability: While the study demonstrates promise, further validation on larger and more diverse datasets would be necessary to confirm the generalizability of these findings.
Summary updated