TL;DR
This systematic review of 87 machine learning (ML) studies reveals that AI/ML approaches show significant promise in predicting suicidal behaviors, achieving high accuracy rates (often >90%) and Area Under the Curve (AUC) values. While the field is rapidly growing, standardization in reporting and methodology is needed to fully realize AI's potential in suicide prevention.
What Problem Does This Paper Address?
Suicide is a major public health crisis, characterized by its complex, transdiagnostic nature and its relative rarity at the population level, making prediction extremely challenging. Despite existing prevention strategies, suicide rates remain stubbornly high and have even increased in some regions. Traditional risk assessment methods often lack sensitivity and clinical utility. This paper addresses the need for more effective suicide risk detection and prevention by systematically reviewing how Artificial Intelligence (AI) and Machine Learning (ML) are being applied to large datasets to improve prediction accuracy.
Methods
The authors conducted a systematic literature search across multiple databases (PubMed/MEDLINE, PsychInfo, Web-of-Science, EMBASE) using keywords related to AI, machine learning, and suicide. They supplemented this with hand-searching and Google Scholar. Inclusion criteria focused on original journal articles in English that employed AI/ML methodologies to evaluate suicide risk outcomes. A PRISMA flow diagram was used to track the study selection process. Data extraction involved coding for study characteristics, ML methodology (supervised vs. unsupervised learning, specific algorithms), and performance metrics (accuracy, AUC, sensitivity, specificity). Descriptive analyses were used to synthesize the findings, as meta-analysis was not possible due to methodological heterogeneity.
Datasets
The paper does not introduce new datasets. Instead, it reviews studies that utilized various existing datasets, including:
- Electronic Medical Records (EMR)
- Administrative chart data
- Epidemiologic surveys
- Social media data (e.g., Twitter, microblogs)
- Clinical samples (e.g., from high-risk or triage settings like emergency departments)
- Archival datasets
The total patient sample size across the included studies was substantial, aggregating to 5,986,238 patients, though sample size was unreported in 7 studies.
Key Insights and Ideas
- AI/ML is a rapidly growing field for suicide prevention: Nearly half of the reviewed studies were published in the last three years, indicating a significant surge in research interest.
- High predictive performance is achievable: Many studies reported impressive accuracy (>90%) and AUC values, suggesting ML models can effectively identify individuals at risk.
- Supervised learning dominates: The majority of studies employed supervised learning techniques, with ensemble methods, naive Bayes, decision trees, and SVMs being common. Unsupervised learning was used less frequently.
- NLP offers unique insights: Natural Language Processing (NLP) was employed in 29 investigations, particularly for analyzing text from social media and clinical notes, showing promise in detecting subtle linguistic cues related to suicide risk.
- Transdiagnostic approach is prevalent: Most studies focused on a broad range of individuals rather than specific psychiatric conditions, reflecting the transdiagnostic nature of suicide.
- Data sources are diverse: ML models are being trained on a wide array of data, from structured EMR data to unstructured social media posts, highlighting the versatility of AI.
Key Results
- 87 studies were included in the qualitative synthesis, with 13 forming a subset focused on NLP analysis of suicide notes.
- High performance metrics: For classification studies (n=65), the average AUC was 0.814 (SD=0.110) and average accuracy was 0.813 (SD=0.123).
- Outcome variability: Studies assessed various suicide outcomes, including suicide attempt (n=28), suicide death (n=14), suicidal ideation (n=9), multiple risk outcomes (n=18), and other-social media (n=10) or undifferentiated outcomes (n=8).
- No significant difference in performance by outcome type: An exploratory ANOVA found no significant differences in accuracy or AUC based on the broad suicide outcome groupings.
- NLP for suicide notes: A subset of 13 studies using NLP on suicide decedent notes showed promising results, with precision values often above 0.60.
Limitations and Future Work
- Methodological heterogeneity: The review highlighted significant variation in ML methods, study quality, and outcome definitions, making direct comparisons and meta-analysis challenging.
- Underreporting of key metrics: Parameters like Positive Predictive Value (PPV), Area Under the Precision Curve (AUPRC), and lead-time of prediction were often underreported.
- Cross-sectional designs dominate: The majority of studies were cross-sectional, underscoring the need for more prospective, longitudinal designs to advance prediction and intervention.
- Generalizability concerns: While many studies used large datasets, concerns about overfitting exist for studies with high-dimensional data and small sample sizes. Replication in new settings and populations is crucial.
- Data limitations: Challenges with EMR data (e.g., incomplete records, biases in collection) and the low correspondence between ICD codes and clinical notes for suicide risk were noted.
- Future directions: The authors emphasize the need for improved reporting standards, a priori-designed studies, replication of successful models, integration of ML into clinical workflows, and a focus on clinical need to guide ML applications. They also suggest exploring the combination of biological, behavioral, and clinical variables for a more comprehensive approach.
Summary updated