TL;DR
Researchers developed XGBoost-based machine learning models to predict the use of mechanical and composite restraints in psychiatric hospitals using electronic health record (EHR) data. While broadening the prediction target to include multiple types of restraint didn't improve the model's ability to pick out specific mechanical restraint cases, it significantly increased the number of at-risk patients identified, offering a more practical tool for clinical prevention.
What Problem Does This Paper Address?
In psychiatric care, coercive measures like mechanical restraint (being tied to a bed) are used as a last resort to prevent harm. However, these measures are ethically fraught and linked to physical and psychological trauma. The clinical goal is always prevention.
The challenge for machine learning is that mechanical restraint is a "rare event." In this study's dataset, it occurred in only about 0.1% of the observation days. This extreme class imbalance makes it difficult to train models that are both accurate and useful. The authors investigated whether training a model on a broader "composite" outcome—including manual and chemical restraints—could provide a stronger training signal and improve the prediction of the most severe cases.
Methods
The team employed a "landmark model" framework for dynamic prediction. Instead of one prediction per admission, the model generates a risk score every morning at 6:00 a.m. for every day a patient is hospitalized, looking ahead 48 hours.
Key technical details include:
- Algorithm: XGBoost, chosen for its efficiency with structured tabular data and its ability to handle missing values natively.
- Feature Engineering: The models utilized 2,389 predictors. These included structured data (diagnoses, demographics, medications) and unstructured clinical notes processed via TF-IDF (Term Frequency-Inverse Document Frequency) to turn free-text observations into numerical features.
- Validation: They used an 85/15 train-test split, with 5-fold cross-validation on the training set to tune hyperparameters.
Datasets
The study used the PSYCOP (PSYchiatric Clinical Outcome Prediction) cohort, a massive longitudinal dataset from the Central Denmark Region.
- Scope: Data from 2015 to 2021.
- Scale: 16,834 unique patients and 45,179 inpatient stays, totaling 687,388 "prediction days."
- Events: 2,736 days were followed by some form of restraint, with 983 specifically involving mechanical restraint.
Key Insights and Ideas
- The "Problem Framing" Effect: Surprisingly, training on a broader composite outcome (manual + chemical + mechanical) did not improve the AUROC for predicting mechanical restraint specifically. However, it did increase the Positive Predictive Value (PPV) for the composite outcome (10.4% vs 4.9%).
- Shared Risk Factors: The high performance of both models suggests that different types of restraint share underlying risk signatures. Identifying a patient at risk for any restraint is often more clinically valuable than only identifying those at risk for one specific type.
- The Power of the Present: Prediction performance was significantly higher the closer the model got to the actual event. This suggests that while long-term forecasting is difficult, "just-in-time" interventions are highly feasible.
- Clinical Utility vs. Raw Accuracy: The authors emphasize that a model with a PPV of 10% (meaning 1 in 10 "high risk" flags is a true positive) is still a 25-fold improvement over random chance and could be a powerful tool for allocating preventive resources.
Key Results
- Mechanical Restraint Model: Achieved an AUROC of 0.921. At a high-specificity threshold (99%), it caught about 30% of restraint cases.
- Composite Restraint Model: Achieved an AUROC of 0.900 for predicting any restraint type.
- Predictor Importance: The Brøset Violence Checklist (BVC) scores were the most influential features. For the composite model, specific keywords in clinical notes like "feel," "home visit," and "woman" also emerged as significant predictors.
- Minimalist vs. Full Model: A "minimal" model using only 17 hand-picked clinical variables still performed well (AUROC 0.87), but the full model with 2,000+ variables provided a statistically significant boost.
Limitations and Future Work
- External Validation: The model was trained and tested on data from a single Danish region. Because legislation and clinical practices regarding restraint vary wildly by country, the model needs testing in different geographical and legal contexts.
- The "Self-Fulfilling Prophecy" Risk: The authors warn that a "high risk" flag could be misinterpreted by staff as a "high need" for restraint, potentially leading to the very outcome the model is meant to prevent. Future work must focus on how to integrate these scores into ward culture safely.
- Fairness Audits: The study did not include a formal fairness audit to check for biases across demographic groups (e.g., ethnicity or gender), which the authors identify as a critical next step.
Summary updated