TL;DR
This paper introduces the "SW-model," a novel, interpretable risk assessment tool for venous thromboembolism (VTE) in surgical patients, developed using real-world data from Southwest China. The SW-model significantly outperforms the traditional Caprini model in VTE risk prediction and offers a more practical, transparent alternative to complex machine learning models like Random Forest.
What Problem Does This Paper Address?
Venous thromboembolism (VTE), a serious condition involving blood clots in veins, is a significant concern for surgical patients. Accurate risk assessment is crucial for timely intervention and preventing complications. The widely used Caprini model, while established, has limitations, particularly in Asian populations, where it tends to overestimate VTE risk, leading to unnecessary treatments and increased healthcare costs. Existing machine learning approaches, while potentially more accurate, often suffer from a "black box" problem, lacking the interpretability needed for clinical trust and adoption. This paper aims to bridge this gap by developing a VTE risk assessment model that is both accurate and clinically interpretable, specifically tailored for surgical patients in Southwest China.
Methods
The researchers developed an improved VTE risk assessment model, dubbed the "SW-model," using Logistic Regression. This approach was chosen for its inherent interpretability, allowing clinicians to understand why a patient is assigned a certain risk score. The model was built and validated using a large real-world dataset of surgical patients. Feature engineering involved selecting relevant risk factors, including those from the Caprini model and additional factors identified from previous research. The model's performance was compared against the Caprini model and a Random Forest model, a popular machine learning algorithm. Evaluation metrics included Area Under the Receiver Operating Characteristic Curve (AUC), sensitivity, specificity, and Decision Curve Analysis (DCA) to assess clinical utility.
Datasets
The study utilized a dataset of 81,505 surgical patients from Southwest Hospital in China, collected between January 1, 2019, and June 18, 2021. This dataset was split into:
- Training dataset: Used to build the SW-model (patients discharged from 2019-2020).
- Retrospective test dataset: Used for initial validation (a random 20% of patients from 2019-2020).
- Prospective test dataset: Used for further validation with more recent data (patients discharged in 2021).
This real-world dataset, encompassing a large cohort of surgical patients, serves as a novel benchmark for VTE risk assessment in this specific demographic and region.
Key Insights and Ideas
- The Caprini model overestimates risk in this population: The study found that a significant majority (86%) of surgical patients in Southwest China were classified as medium, high, or highest risk by the Caprini model, indicating a substantial overestimation of VTE risk. This highlights the need for a more tailored approach.
- Interpretability is key for clinical adoption: While machine learning models like Random Forest showed higher accuracy, their "black box" nature hinders clinical trust. The SW-model, built on Logistic Regression, offers a transparent and understandable risk assessment, crucial for practical implementation.
- A simpler model can be effective: The SW-model incorporates only eight risk factors, making it significantly easier to use in clinical practice compared to the Caprini model, which relies on a much larger set of factors. This simplicity reduces the burden of data collection and processing.
- COVID-19's indirect impact: The study acknowledges the potential indirect influence of the COVID-19 pandemic on VTE incidence due to changes in patient behavior and healthcare access, suggesting this as a factor to consider in future analyses.
Key Results
- Improved Discriminative Capability: The SW-model demonstrated significantly superior AUC values compared to the Caprini model on both retrospective (0.807 vs. 0.705) and prospective (0.804 vs. 0.758) test sets (p=0.001 and p=0.044, respectively).
- Comparison with Random Forest: While the SW-model was inferior to Random Forest in AUC (0.804 vs. 0.839 on the prospective test set), it offered crucial interpretability.
- Clinical Utility: Decision Curve Analysis (DCA) showed that the SW-model provided better clinical benefits than the Caprini model and default strategies within a specific threshold range, indicating its practical value in guiding treatment decisions.
- Stratification Accuracy: The SW-model effectively stratified patients into distinct risk groups with significant differences in VTE incidence, unlike the Caprini model, which tended to group most patients into the highest risk category.
Limitations and Future Work
The authors acknowledge several limitations:
- Single-center study: The data was collected from one hospital, limiting its generalizability to the broader Chinese surgical population.
- In-hospital VTE focus: The study only considered VTE cases occurring during hospitalization, potentially underestimating the total VTE burden if cases occurring shortly after discharge were missed.
Future work should include:
- Multi-center validation: Conducting larger, multi-center studies to validate the SW-model's applicability across diverse populations.
- Broader VTE definition: Incorporating VTE events occurring within a defined period after discharge to provide a more comprehensive risk assessment.
- Further refinement: Exploring variants of the SW-model or other interpretable machine learning techniques to further enhance predictive accuracy while maintaining clinical transparency.
Summary updated