TL;DR
This paper introduces an "adaptive-mixture-categorization" (AMC) method to better extract suicide risk predictors from unstructured clinical notes in electronic health records (EHRs). By transforming skewed and zero-inflated natural language processing (NLP) variables, AMC significantly improves their association with suicide risk compared to original or standard quantile-based methods, offering a more robust approach to suicide risk assessment.
What Problem Does This Paper Address?
Suicide is a major public health crisis, and accurately identifying individuals at risk is a critical challenge. Traditional methods rely on clinical assessments and self-reports, which can be limited. Natural Language Processing (NLP) offers a way to extract valuable information from the vast amount of unstructured text in Electronic Health Records (EHRs), such as clinical notes. However, NLP-derived variables often suffer from "zero inflation" (many zeros) and skewed distributions, making them difficult to use effectively in predictive models. Standard statistical methods can produce biased estimates or increase model variance when dealing with such data. This paper aims to address this challenge by developing a preprocessing method that optimizes the predictive power of these NLP variables for suicide risk.
Methods
The core innovation in this paper is the Adaptive-Mixture-Categorization (AMC) method. This data-driven approach transforms continuous, zero-inflated, and skewed NLP variables into categorical variables. Unlike traditional methods that pre-define categories (e.g., quantiles), AMC uses an F-statistic-based approach to find optimal thresholds that maximize the variance between categories while minimizing variance within categories. This process aims to retain as much predictive information as possible.
The authors also implemented an efficient search strategy for AMC to handle the computational burden of large datasets. This involved focusing the search on larger gaps in the data, significantly reducing computation time.
For comparison, they also used:
- Original NLP variables: Treated as continuous variables.
- Quantile-categorized NLP variables: Standard method of dividing variables into equal-sized groups based on their distribution.
The association strength between these preprocessed NLP variables and suicide risk was assessed using chi-square tests and logistic regression for linear trends.
Datasets
The study utilized data from the U.S. Department of Veterans Affairs (VA).
- Suicide Cohort: 4,990 VA patients who died by suicide between 2015-2016.
- Control Group: 20,352 randomly matched VA patients who did not die by suicide during the same period.
- Clinical Notes: A combined corpus of 1,336,187 notes from the VA's Corporate Data Warehouse (CDW) and the VA/Department of Defense (VA/DoD) Mortality Data Repository.
The NLP variables were generated using the SÉANCE (Sentiment Analysis and Social Cognition Engine) package, which leverages 8 linguistic databases.
Key Insights and Ideas
- Zero-inflation and skewness are major hurdles: The paper clearly articulates why these characteristics of NLP data make direct modeling difficult and lead to suboptimal results.
- AMC unlocks hidden predictive power: The core insight is that by intelligently categorizing NLP variables, AMC can reveal stronger associations with suicide risk than simply using the raw variables or standard quantile methods.
- Data-driven categorization is key: The adaptive nature of AMC, which finds optimal thresholds based on the data itself, is crucial for its success.
- Interpretability can be maintained: While AMC creates more categories, the paper suggests that treating these categories as ordinal or even continuous (via linear trend tests) can preserve interpretability for clinicians.
- Robustness across analyses: The improved associations with AMC held true in both the full cohort analysis and sensitivity analyses using undersampled data, suggesting reliability.
Key Results
- Significantly stronger associations: AMC-categorized NLP variables showed substantially stronger associations with suicide risk compared to original continuous NLP variables and quantile-categorized NLP variables.
- Magnitude of improvement: In terms of significance (mean -log10 p-values), AMC-L (ordinal AMC) achieved 130.3, significantly outperforming Quantile-L (112.8) and untransformed variables (35.9).
- Effect size enhancement: AMC categorization nearly doubled the average Cohen's D effect size compared to original NLP variables. It also outperformed quantile categorization by approximately 15% in average effect size.
- Improved risk stratification: Visualizations (Fig. 4) show that AMC effectively captures linearly increasing risk in the right-tail region, whereas quantile categorization tends to consolidate high-risk values, losing granularity.
- Sensitivity analysis confirms findings: In undersampled scenarios (5000 participants), AMC-categorized variables outperformed quantile-categorized variables over 90% of the time, with a 15% greater average effect size.
Limitations and Future Work
- Generalizability: The findings are primarily based on VA patient data and may not be fully generalizable to populations outside the VA system. Future studies should validate these findings in diverse clinical settings.
- Computational Cost: While an efficient search strategy was implemented, AMC categorization still has a significant computational cost, especially for large-scale deployment. Future work could explore further optimizations or parallel computing.
- Interpretability of Data-Driven Thresholds: The data-driven thresholds of AMC are less intuitive than standard quantiles. The authors suggest treating categories as ordinal or continuous to mitigate this.
- Category Optimization: The study limited the search to a maximum of 10 categories. Future work could explore increasing this limit to potentially capture more nuanced risk.
- Refining AMC: Future research plans include refining the AMC method for determining the optimal number of categories and exploring the predictive performance of NLP variables preprocessed by AMC.