Development and Evaluation of Machine Learning Models to Predict Mechanical Restraint and Related Coercive Measures in Hospital Psychiatry

Sara Kolding, Jakob Grøhn Damgaard, Martin Bernstorff, Lasse Hansen, Søren Dinesen Østergaard, Andreas Aalkjær DanielsenmedRxiv 2025 · DOI

TL;DR

Researchers developed XGBoost-based machine learning models to predict the use of mechanical and composite restraints in psychiatric hospitals using electronic health record (EHR) data. While broadening the prediction target to include multiple types of restraint didn't improve the model's ability to pick out specific mechanical restraint cases, it significantly increased the number of at-risk patients identified, offering a more practical tool for clinical prevention.

What Problem Does This Paper Address?

In psychiatric care, coercive measures like mechanical restraint (being tied to a bed) are used as a last resort to prevent harm. However, these measures are ethically fraught and linked to physical and psychological trauma. The clinical goal is always prevention.

The challenge for machine learning is that mechanical restraint is a "rare event." In this study's dataset, it occurred in only about 0.1% of the observation days. This extreme class imbalance makes it difficult to train models that are both accurate and useful. The authors investigated whether training a model on a broader "composite" outcome—including manual and chemical restraints—could provide a stronger training signal and improve the prediction of the most severe cases.

Methods

The team employed a "landmark model" framework for dynamic prediction. Instead of one prediction per admission, the model generates a risk score every morning at 6:00 a.m. for every day a patient is hospitalized, looking ahead 48 hours.

Key technical details include:

  • Algorithm: XGBoost, chosen for its efficiency with structured tabular data and its ability to handle missing values natively.
  • Feature Engineering: The models utilized 2,389 predictors. These included structured data (diagnoses, demographics, medications) and unstructured clinical notes processed via TF-IDF (Term Frequency-Inverse Document Frequency) to turn free-text observations into numerical features.
  • Validation: They used an 85/15 train-test split, with 5-fold cross-validation on the training set to tune hyperparameters.

Datasets

The study used the PSYCOP (PSYchiatric Clinical Outcome Prediction) cohort, a massive longitudinal dataset from the Central Denmark Region.

  • Scope: Data from 2015 to 2021.
  • Scale: 16,834 unique patients and 45,179 inpatient stays, totaling 687,388 "prediction days."
  • Events: 2,736 days were followed by some form of restraint, with 983 specifically involving mechanical restraint.

Key Insights and Ideas

  • The "Problem Framing" Effect: Surprisingly, training on a broader composite outcome (manual + chemical + mechanical) did not improve the AUROC for predicting mechanical restraint specifically. However, it did increase the Positive Predictive Value (PPV) for the composite outcome (10.4% vs 4.9%).
  • Shared Risk Factors: The high performance of both models suggests that different types of restraint share underlying risk signatures. Identifying a patient at risk for any restraint is often more clinically valuable than only identifying those at risk for one specific type.
  • The Power of the Present: Prediction performance was significantly higher the closer the model got to the actual event. This suggests that while long-term forecasting is difficult, "just-in-time" interventions are highly feasible.
  • Clinical Utility vs. Raw Accuracy: The authors emphasize that a model with a PPV of 10% (meaning 1 in 10 "high risk" flags is a true positive) is still a 25-fold improvement over random chance and could be a powerful tool for allocating preventive resources.

Key Results

  • Mechanical Restraint Model: Achieved an AUROC of 0.921. At a high-specificity threshold (99%), it caught about 30% of restraint cases.
  • Composite Restraint Model: Achieved an AUROC of 0.900 for predicting any restraint type.
  • Predictor Importance: The Brøset Violence Checklist (BVC) scores were the most influential features. For the composite model, specific keywords in clinical notes like "feel," "home visit," and "woman" also emerged as significant predictors.
  • Minimalist vs. Full Model: A "minimal" model using only 17 hand-picked clinical variables still performed well (AUROC 0.87), but the full model with 2,000+ variables provided a statistically significant boost.

Limitations and Future Work

  • External Validation: The model was trained and tested on data from a single Danish region. Because legislation and clinical practices regarding restraint vary wildly by country, the model needs testing in different geographical and legal contexts.
  • The "Self-Fulfilling Prophecy" Risk: The authors warn that a "high risk" flag could be misinterpreted by staff as a "high need" for restraint, potentially leading to the very outcome the model is meant to prevent. Future work must focus on how to integrate these scores into ward culture safely.
  • Fairness Audits: The study did not include a formal fairness audit to check for biases across demographic groups (e.g., ethnicity or gender), which the authors identify as a critical next step.

Summary updated

Abstract

Abstract Introduction Use of coercive measures in psychiatric hospitals is clinically and ethically challenging. Aiming to support prevention, we developed and evaluated machine learning models to predict both mechanical restraint and a broader composite outcome that includes related coercive measures. Methods The dataset comprised electronic health records (EHR) from adults (≥18 years) who had at least one admission to the Psychiatric Services in the Central Denmark Region between 2015 and 2021. For each inpatient day, XGBoost machine learning models were trained to predict mechanical restraint or composite (mechanical, chemical, or manual) restraint within 48 hours. Hyperparameters were optimised for the area under the receiver operating characteristic curve (AUROC) using five-fold cross validation on 85% of the data, with performance validated on a held-out 15% test set. Results The cohort included 16,834 patients with 45,179 inpatient stays, covering 687,388 prediction days. Of these, 2,736 days were followed by a restraint episode within 48 hours, including 983 episodes of mechanical restraint. The final models were trained on 2,389 EHR-based predictors, derived from demographics, diagnoses, medications, and clinical notes. The mechanical restraint model achieved an AUROC of 0.921 (95% CI: [0.918–0.922]) and a positive predictive value (PPV) of 4.9% when classifying the top 1% of risk scores as positive. The composite model achieved an AUROC of 0.912 (95% CI: [0.909–0.913]) and a PPV of 4.2% when predicting mechanical restraint, and 0.900 (95% CI: [0.898–0.900]) with a PPV of 10.4% when predicting composite restraint. Conclusion The results indicate that incorporating related coercive measures into model training did not improve discrimination (AUROC) for predicting mechanical restraint but did increase PPV when predicting composite restraint, reflecting the higher outcome prevalence. This suggests that leveraging related outcomes can inform prediction of rare events, emphasising the importance of problem framing in clinical prediction modelling. Future work should include external validation across temporal, geographic, and demographic contexts. Significant Outcomes: – A machine learning model trained solely for predicting mechanical restraint achieved strong performance (AUROC 0.92), identifying nearly one-third of restraint cases at high specificity. – Training on a broader composite outcome yielded similar discriminatory performance when predicting mechanical restraint, while the higher base rate resulted in a higher positive predictive value for predicting composite restraint. – Broadening the outcome to include multiple restraint types increased the number of at-risk patients detected due to the higher prevalence, without compromising accuracy for mechanical restraint, supporting shared underlying risk factors. Limitations – The model requires more extensive external validation to assess generalisability across time, demographic groups, and settings, which may be limited by regional/national differences in legislation and clinical documentation. – Prediction performance was highest near the restraint event, limiting early forecasting and suggesting that limiting predictions to the early phase of hospitalisation, where most restraint occurs, could elevate the base rate and improve model performance.
BibTeX
@misc{kolding2025development,
  author = {Sara Kolding and Jakob Grøhn Damgaard and Martin Bernstorff and Lasse Hansen and Søren Dinesen Østergaard and Andreas Aalkjær Danielsen},
  title = {{Development and Evaluation of Machine Learning Models to Predict Mechanical Restraint and Related Coercive Measures in Hospital Psychiatry}},
  year = {2025},
  doi = {10.64898/2025.12.15.25342272},
  howpublished = {medRxiv},
}
Figure