Multidimensional Suicide Risk Assessment with Machine Learning using Electronic Health Records

Nathan Scollar, Kemal Davaslioglu, Nathan Conger, Sastry Kompella, Yalin E. Sagduyu, Nathaniel D. BastianarXiv 2026 · arXiv

TL;DR

Researchers have developed a multidimensional machine learning framework using Electronic Health Records (EHR) to retrospectively identify suicide-related risk. By combining a novel age-matched resampling strategy with a calibrated XGBoost model, the team achieved an F1 score of 0.81, highlighting that diagnostic complexity and social determinants are just as critical as psychiatric history in assessing risk.

What Problem Does This Paper Address?

Suicide prevention is a massive public health challenge, with nearly 50,000 deaths in the U.S. in 2023 alone. While EHRs contain a treasure trove of clinical and behavioral data, building predictive models from them is notoriously difficult due to two main issues:

  1. Severe Class Imbalance: Suicide-related events are (thankfully) rare in the general population, often making up less than 5% of a dataset. Standard models often struggle to learn from such "needle-in-a-haystack" scenarios.
  2. Age-Related Confounding: Health profiles and suicide prevalence change drastically with age. Without careful control, a model might inadvertently learn to use age as a proxy for risk rather than actual clinical indicators.

The authors aim to create a robust, interpretable framework that integrates psychiatric, medical, and social factors to better characterize high-risk patients.

Methods

The authors proposed a three-pronged approach:

  • Feature Engineering: They mapped over 27,000 unique clinical diagnoses into 53 clinically motivated categories (shown in Fig. 1). These span psychiatric conditions, substance abuse, and social determinants of health (SDoH).
  • Age-Matched Resampling: To combat class imbalance and confounding, they implemented a 3:1 matching strategy. For every patient with a suicide-related outcome, they sampled three "control" patients from the same age group. This ensured a consistent 25% positive rate across the analytical cohort.
  • Model Pipeline: They evaluated Logistic Regression, Random Forests, and XGBoost. Crucially, they applied isotonic calibration to the models. This step aligns the model's predicted probabilities with actual observed frequencies, which is vital for clinical utility.
Fig. 1. The 53 health-related categories utilized for creating features.

Datasets

The study utilized MIMIC-IV, a massive, de-identified EHR dataset containing records from over 200,000 patients admitted to the emergency department or intensive care units at Beth Israel Deaconess Medical Center. The final dataset included over 6.3 million recorded diagnoses.

Key Insights and Ideas

  • The "Diagnostic Burden" Proxy: One of the most significant predictors was simply the total count of a patient's diagnoses. This suggests that overall medical complexity and frequent healthcare utilization are strong indicators of underlying risk.
  • SDoH are High-Impact: While depression is a known factor, social determinants like exposure to violence and housing instability showed massive univariate associations (Odds Ratios of 34.6 and 16.6, respectively).
  • Calibration is Essential: Raw machine learning scores often over- or under-estimate risk. The authors showed that isotonic calibration significantly improved the reliability of predicted probabilities, making the model's output more "honest."
Fig. 2. The impact of isotonic model calibration.

Key Results

  • Top Performer: The calibrated XGBoost model was the clear winner, achieving an F1 score of 0.81 and an Area Under the Precision-Recall Curve (AUPRC) of 0.89 on the age-matched cohort.
  • Feature Importance: SHAP (Shapley Additive Explanations) analysis revealed that while "has_Depression" was the most influential feature, personality disorders, SDoH-related violence, and even specific age indicators (like the 76-85 bracket) played major roles in the model's decisions.
Fig. 4. Top 15 XGBoost feature importance values for classification of the composite suicide-related outcome after age-matched resampling.
  • Robustness: The model maintained stable performance across 20 different random resampling seeds, suggesting the findings aren't just a fluke of the specific patients chosen for the control group.

Limitations and Future Work

  • Retrospective Bias: The study relies on documented ICD codes. Patients who struggle with suicidal ideation but never receive a formal diagnosis code for it would be misclassified as "low risk" in this training set.
  • Elderly Performance Gap: The model's recall dropped significantly for patients over 76. The authors attribute this to smaller sample sizes in older cohorts and suggest that geriatric suicide risk may require specialized modeling.
  • Next Steps: The authors plan to move from static "snapshot" modeling to temporal sequence modeling, capturing how a patient's risk evolves over time across multiple hospital visits.
Fig. 5. Distribution of SHAP values for total diagnoses among high-scoring patients in the held-out control cohort.

Summary updated

Abstract

Suicide prevention remains a critical public health challenge, and electronic health records (EHRs) provide an opportunity to characterize suicide-related risk indicators using documented clinical, behavioral, medical, and social-contextual information. This paper presents a multidimensional machine learning framework to model and estimate suicide-related outcomes using MIMIC-IV EHR data. We construct clinically motivated diagnosis categories and define a composite outcome from documented codes corresponding to suicidal ideation, suicide attempt, prior suicidal behavior, and intentional self-harm. To address severe class imbalance and reduce age-related confounding, we apply age-matched resampling that retains all positive patients and samples comparison-cohort patients from the same age group. Logistic regression, random forest, and XGBoost classifiers are evaluated before and after resampling with isotonic calibration. The calibrated XGBoost model achieves the strongest matched-cohort performance, outperforming the other evaluated classifiers. SHAP-based interpretation shows that diagnostic burden, depression-related features, age, personality-disorder history, social determinants related to violence, and medical comorbidity contribute to elevated predicted probabilities. Our results demonstrate that combining broad EHR-derived features, age-matched cohort construction, calibration, and feature-level interpretation provides a practical framework for retrospective suicide-related risk modeling.
BibTeX
@misc{scollar2026multidimensional,
  author = {Nathan Scollar and Kemal Davaslioglu and Nathan Conger and Sastry Kompella and Yalin E. Sagduyu and Nathaniel D. Bastian},
  title = {{Multidimensional Suicide Risk Assessment with Machine Learning using Electronic Health Records}},
  year = {2026},
  eprint = {2611.03357},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03357},
}
Figure