Introduction
Diagnosing severe mental illnesses, specifically schizophrenia and bipolar I disorder, remains a significant clinical challenge. Differential diagnosis—distinguishing between the two—is particularly difficult, with research suggesting that up to 30% of bipolar patients are initially misdiagnosed. While "thought disorders" manifested through incoherent speech are recognized hallmarks, traditional assessments rely on subjective clinical interviews. This study introduces an automated Natural Language Processing (NLP) pipeline designed to provide an objective, interpretable assessment of social functional competence, anchored to the validated Social Skills Performance Assessment (SSPA).
Methodology
The researchers developed a methodology focused on linguistic features that correlate with social skills rather than a purely data-driven "black box" approach. The pipeline involved three primary stages:
- Feature Extraction: 73 features were extracted across two categories:
- Semantic Coherence: Utilizing word and sentence embeddings (Word2Vec, Smooth Inverse Frequency, and InferSent) to measure the flow of meaning between interviewer and subject.
- Linguistic Complexity: Including lexical diversity (vocabulary variety), lexical density (content-to-filler word ratio), and syntactic complexity (sentence structural depth).
- SSPA Modeling: A greedy stepwise search identified a subset of 25 features best suited to predict manual SSPA scores assigned by human raters.
- Classification: Logistic Regression and Naive Bayes classifiers were trained to distinguish clinical subjects from healthy controls and to perform differential diagnosis between schizophrenia and bipolar disorder.
The study utilized transcripts from 109 subjects (43 Schizophrenia/Schizoaffective, 44 Bipolar I, and 22 Healthy Controls) performing SSPA role-playing tasks.
Results
The automated system demonstrated strong alignment with human clinical judgment. The linear regression model predicted human-assigned SSPA scores with a correlation coefficient of 0.752, as shown in Figure 1.
In classification tasks, the models achieved high accuracy. Distinguishing clinical subjects from healthy controls yielded an Area Under the Curve (AUC) of 0.96. The more challenging task of differential diagnosis between schizophrenia and bipolar disorder achieved an AUC of 0.826 using a Naive Bayes classifier.
Key Insights
- The "Stress Test" of Negotiation: Features from the "negotiating with a landlord" scene (Scene 3) were the most predictive. This suggests that demanding social interactions involving conflict resolution are more effective at revealing subtle language impairments.
- Interpretability: By grounding the model in features that correlate with known clinical scores (SSPA), the researchers ensured that the AI's outputs are clinically relevant and interpretable.
- Ensemble Power: The most effective feature sets combined simple metrics like "Bag-of-Words" averages with sophisticated deep-learning sentence encodings (InferSent), highlighting the importance of both word choice and complex structure.
Limitations and Future Work
While promising, the study is limited by a relatively small sample size (n=109). Additionally, using the same dataset for feature selection and testing may lead to slightly optimistic results. Future research will focus on refining the feature set for clinical use, subtyping individuals within diagnostic groups, and expanding the analysis to other psychiatric conditions and speech tasks.
Summary updated