TL;DR
This paper introduces a novel deep learning approach for passively diagnosing depression and anxiety by analyzing individuals' written text. By modeling the interconnectedness of symptoms and using a multidimensional classification architecture, the system shows promise in predicting PHQ-4 scores, outperforming random baselines and offering a new avenue for early mental health screening.
What Problem Does This Paper Address?
Depression and anxiety are widespread mental health issues that significantly impact individuals' lives and contribute to global disability. A major challenge in addressing these conditions is that many individuals are unaware of their symptoms or attribute them to general mood fluctuations, delaying or preventing them from seeking professional help. This paper tackles the problem of "passive diagnosis" – using technology to identify potential mental health issues without requiring active patient engagement. Specifically, it aims to predict scores on the PHQ-4, a short questionnaire assessing core symptoms of depression and anxiety, directly from written text. This moves beyond traditional "active diagnosis" where individuals must seek out medical professionals.
Methods
The core of this work is a deep neural network (DNN) architecture designed for multilabel and multidimensional classification. Unlike simpler models that treat each symptom independently, this DNN explicitly models the interconnectivity between the four PHQ-4 symptoms.
The approach involves two main steps:
- Multilabel Learning: A binary cross-entropy loss function and a Sigmoid activation on the final layer are used to predict a probability score for each of the four symptoms. This allows the model to learn how symptoms relate to each other.
- Multivalue (Ordinal) Classification: Manual threshold values are applied to the Sigmoid scores to convert them into ordinal values (0-3) for interpretability, reflecting the severity of each symptom.
Feature extraction is a key component, employing three methods:
- LIWC (Linguistic Inquiry and Word Count): This dictionary-based approach extracts counts of words related to 22 psychological dimensions identified as relevant from previous research.
- Psychometric Similarity: This method uses embeddings from the Universal Sentence Encoder (USE) to compare the semantic similarity between patient statements and questions from relevant psychometric questionnaires. The inner dot product of these embeddings serves as a feature.
- Text Representation: The Universal Sentence Encoder (USE) is used to generate fixed-size (512-dimensional) embeddings for each patient statement, capturing semantic meaning.
Hyperparameter tuning was performed using a genetic algorithm to find optimal network configurations.
Datasets
- DAIC-WOZ: This dataset, collected in an in-person medical setting, consists of transcribed clinical interviews from 142 patients. It includes patient statements and PHQ-8 scores (from which PHQ-2 and GAD-2 scores were inferred). This dataset was used for training and evaluating the model, offering a crucial advantage over social media data by providing clinically validated scores.
- Reddit Depression Dataset: This dataset, compiled by Losada and Crestani (2016), was used for evaluating the model's performance on social media data. It contains Reddit posts from 253 users, with a binary classification of depressed or non-depressed based on their participation in specific sub-forums.
Key Insights and Ideas
- Interconnectivity of Symptoms: The paper strongly advocates for modeling the interconnectedness of depression and anxiety symptoms, rather than treating them as independent issues. This is reflected in their choice of a multilabel and multidimensional DNN architecture.
- Passive Diagnosis Potential: The work demonstrates that analyzing written text can indeed predict psychometric scores, offering a viable path towards passive diagnosis and early intervention for mental health conditions.
- Domain Shift Challenge: Evaluating the model trained on clinical interview data on social media data revealed a significant performance drop. This highlights the non-trivial challenge of domain shift and the need for models that can generalize across different writing styles and contexts.
- Early Diagnostics vs. Specificity: The authors note that for early-stage diagnostics, high sensitivity (minimizing false negatives) is often prioritized over high specificity, even if it means a slightly higher rate of false positives.
Key Results
- Multilabel Evaluation: The proposed multilabel DNN achieved a Hamming loss of 0.388 (95% CI [0.3870, 0.3905]), significantly outperforming a random baseline with a Hamming loss of 0.49 (95% CI [0.481, 0.519]).
- Multidimensional Evaluation: The model achieved an Example Accuracy of 0.221 (95% CI [0.201, 0.243]), which was a 14% improvement over a random forest classifier (0.087, 95% CI [0.086, 0.085]).
- Social Media Evaluation: When evaluated on the Reddit dataset, the model achieved an accuracy of 43%, which was 18% below the majority class baseline, underscoring the domain shift challenge.
- Sensitivity: The model demonstrated high sensitivity across all four PHQ-4 questions, indicating a good ability to identify individuals who might be experiencing symptoms.
Limitations and Future Work
- Domain Generalization: The significant performance drop when evaluating on social media data indicates a limitation in the model's ability to generalize across different domains. Future work should focus on developing models that are less domain-specific and can better adapt to various writing styles. Transfer learning is suggested as a potential avenue.
- Temporal Aspect: The current approach does not consider the temporal aspect of prediction, unlike some other works in the eRisk workshop. Future research could explore incorporating time-series analysis.
- Generalizability: The authors acknowledge that further work is required to ensure the generalizability of the approach.
- Feature Set Exploration: While three feature sets were used, exploring other feature extraction methods could further enhance performance.
Summary updated