TL;DR
This paper presents a machine learning classifier that effectively identifies individuals with Familial Hypercholesterolemia (FH) using electronic health record (EHR) data. By leveraging a random forest model, the system achieves high accuracy in flagging potential FH cases, paving the way for more efficient and targeted screening, ultimately aiming to reduce the significant burden of undiagnosed FH and its associated cardiovascular risks.
What Problem Does This Paper Address?
Familial Hypercholesterolemia (FH) is a common genetic disorder that significantly increases the risk of premature coronary artery disease. Despite its prevalence and the availability of effective treatments, FH remains severely underdiagnosed, with estimates suggesting less than 10% of affected individuals in the US are identified. Current diagnostic approaches, relying on clinical criteria like DLCN or Simon-Broome, have limitations: they are often non-specific (many individuals with high LDL-C don't have FH), and they depend heavily on the availability of untreated lipid levels and detailed family histories, which are frequently incomplete or inaccessible to healthcare providers. This diagnostic bottleneck prevents timely intervention and cascade screening of relatives, leading to preventable morbidity and mortality. The paper aims to bridge this gap by developing a data-driven method to proactively identify individuals at high risk for FH within large healthcare systems.
Methods
The core of this research is a random forest classifier trained on electronic health record (EHR) data. Random forests were chosen for their resistance to overfitting, interpretability, and ability to rank feature importance. The process involved:
- Feature Engineering: The model utilized both structured (e.g., lab results, diagnosis codes, prescriptions) and unstructured (e.g., clinical notes) EHR data. Lab values were binned into five categories (very low, low, in-range, high, very high) to better capture extreme values characteristic of FH. Text mentions were processed to identify medical terms and flag negated findings.
- Classifier Training: A random forest model was trained using data from known FH patients and matched non-cases. To address the class imbalance inherent in FH prevalence, a 1:30 prevalence was used during training (upsampling FH cases).
- Internal Validation: The classifier's performance was evaluated on a held-out test set from Stanford Health Care.
- External Validation: The trained classifier was then tested on data from the Geisinger Healthcare System, including genetically confirmed FH cases, to assess its generalizability.
- Chart Review: To further validate the classifier's real-world utility, a manual chart review was conducted on 100 patients flagged by the classifier at Stanford, assessing their FH probability against established clinical criteria (DLCN and MEDPED).
Datasets
- Stanford Health Care Dataset: Used for training and internal validation. This dataset integrated data from Stanford Children's Health and Stanford Health Care hospitals, comprising data from 3.1 million patients, including clinical notes, diagnoses, procedures, laboratory tests, and medication orders.
- Training Data: 197 known FH cases and 35,562 potential non-cases.
- Internal Test Set: A subset of the Stanford data, evaluated at a 1:70 prevalence.
- Geisinger Healthcare System Dataset: Used for external validation. This dataset included data from 33,086 patients, with a specific subset of 466 FH cases (236 genetically confirmed) and 5,000 matched non-cases. For evaluation, a 1:70 prevalence subset (71 FH cases and 4,970 non-cases) was used to avoid overly optimistic estimates.
Key Insights and Ideas
- EHR Data is Rich for FH Identification: The study demonstrates that a wealth of information within standard EHRs, including unstructured text, can be effectively leveraged to identify complex genetic conditions like FH, going beyond traditional clinical criteria.
- Feature Diversity is Crucial: The top features used by the classifier included not only direct cholesterol measurements but also mentions of "Triglycerides," "Red meat" (indicating lifestyle factors), and diagnoses like "Mixed hyperlipidemia" and "Paroxysmal supraventricular tachycardia," highlighting the multifactorial nature of risk and the model's ability to capture subtle associations.
- Machine Learning as a Proactive Screening Tool: The research shifts the paradigm from reactive diagnosis based on symptoms to proactive identification of at-risk individuals, enabling earlier intervention and potentially preventing severe cardiovascular events.
- External Validation is Key for Trust: Demonstrating the classifier's performance across two independent healthcare systems (Stanford and Geisinger) significantly bolsters confidence in its generalizability and potential for widespread adoption.
Key Results
- Internal Validation (Stanford):
- Positive Predictive Value (PPV): 0.88
- Sensitivity: 0.75
- Specificity: 0.99
- AUROC: 0.94
- AUPRC: 0.71
- External Validation (Geisinger):
- PPV: 0.85
- Sensitivity: 0.68
- Specificity: 0.99
- AUROC: 0.94
- AUPRC: 0.68
- Chart Review: The classifier correctly flagged 84% of patients at the highest probability threshold (0.99-0.90) as meeting DLCN or MEDPED criteria for FH.
- Cost-Effectiveness: The paper highlights that using this EHR-based screening can drastically reduce the cost and clinician time required to identify a new FH case compared to traditional methods. For instance, the cost to find one new case drops from an estimated $70,000 to $1,429 with EHR-based screening.
Limitations and Future Work
- Limitations:
- Dataset Size: The number of positive FH patients in the Stanford training set (n=197) is relatively small, and the authors anticipate that more data would improve the classifier.
- Prevalence Estimation: The accuracy of the classifier can be influenced by the actual prevalence of FH in a given clinical setting, which was estimated at 1:70 for evaluation.
- Downstream Outcomes: The study did not measure long-term outcomes such as FH diagnosis rates, cascade screening success, or ASCVD event reduction after implementation.
- Pooled Training Data: The classifier was not trained on pooled data from both sites to ensure a robust assessment of external validity.
- Future Work:
- Deployment: The authors are working on deploying the model in clinical settings at Stanford Healthcare and other sites in partnership with the FH Foundation.
- Continuous Refinement: The classifier will be continuously refined as new diagnosed cases become available.
- Broader Application: The approach could be extended to identify other underdiagnosed Mendelian diseases.
- Integration with Clinical Workflows: The next steps involve integrating this tool into existing clinical workflows to facilitate targeted evaluation and intervention.
Summary updated