Abstract
The possibility of diagnosing depression using interview text was confirmed; regarding suicide risk, the diagnosis accuracy increased when demographic variables were incorporated. Therefore, participants' words during an interview show significant potential as an objective and diagnostic marker through machine learning.
Introduction
Diagnosing depression and assessing suicide risk remains a significant challenge in modern medicine, primarily because it relies heavily on subjective self-reporting and clinical intuition. Unlike other medical fields that utilize objective markers—such as blood tests for diabetes or EKGs for heart disease—psychiatry often lacks standardized, objective tools to confirm diagnoses. While previous research has explored social media activity or vocal acoustics, these methods often fail to translate effectively into clinical settings. This study investigates whether the specific vocabulary used during standard, semi-structured psychiatric interviews (the Mini-International Neuropsychiatric Interview, or MINI) contains latent patterns that can serve as objective diagnostic markers.
Methodology
The research team conducted interviews with 166 participants at Seoul National University Hospital, consisting of 83 healthy controls and 83 patients diagnosed with depression. The depressed group was further stratified into high-suicide-risk (n=31) and low-suicide-risk (n=52) subgroups.
The analysis followed a multi-stage machine learning pipeline:
- Speech-to-Text: Audio recordings were transcribed using the Google Speech-to-Text API.
- Natural Language Processing (NLP): Morphological analysis and part-of-speech tagging for the Korean language were performed using the
Konlpypackage. - Classification: A Naive Bayes classifier was used to calculate the probability of a diagnosis based on word frequency.
- Ensemble Modeling: For suicide risk assessment, the researchers developed an ensemble model combining text-based probabilities with demographic data (e.g., age, BMI, medication dosage) modeled via Gaussian distributions.
- Validation: The models were evaluated using 5-fold cross-validation and the DeLong test for statistical significance.
Results and Analysis
The study found that linguistic patterns are highly predictive of a patient's mental state. For depression detection, text-based features significantly outperformed demographic data.
As shown in the ROC curve analysis, the text-based model achieved an impressive Area Under the Curve (AUC) of 0.905, whereas demographics alone yielded an AUC of 0.761. The model demonstrated high specificity (0.964), indicating a strong ability to correctly identify healthy individuals.
The confusion matrix further illustrates the model's accuracy, correctly identifying 80 out of 83 healthy controls and 58 out of 83 patients with depression using text alone.
In contrast, suicide risk prediction proved more complex. Text alone was only moderately successful (AUC 0.632), but the ensemble model—combining text with demographic data—reached an AUC of 0.800. This suggests that while "word choice" is a powerful indicator of depression, suicide risk is better captured by a combination of linguistic markers and personal background factors. Specific "red flag" words identified in the depressed group included terms with negative connotations such as "hospitalization," "mistake," and "negation."
Discussion and Limitations
These findings suggest that "word choice" could serve as a powerful new objective diagnostic marker in psychiatry. However, several limitations must be addressed in future research. The sample size of 166 is relatively small for machine learning, and the cohort was predominantly female (80%), which may introduce bias. Additionally, the current Naive Bayes model focuses on word frequency and may miss nuances like sarcasm or context-dependent reversals of meaning.
Future work should explore more sophisticated deep learning architectures, such as Transformers (e.g., BERT), to capture richer contextual information. Furthermore, while this study used structured interviews, analyzing unstructured "free speech" might provide even deeper insights into a patient's psychological state. Establishing causality—whether language changes because of depression or serves as a precursor—will require longitudinal studies.
Summary updated