Predicting Atrial Fibrillation Ablation Outcomes: A Machine Learning Approach Leveraging a Large Administrative Claims Database

Yijun Liu, Mustapha Oloko-Oba, Kathryn Wood, Michael S. Lloyd, Joyce C. Ho, Vicki HertzbergmedRxiv 2024 · DOI

This paper explores whether machine learning (ML) can improve our ability to predict the success of atrial fibrillation (AF) ablation procedures using large-scale administrative claims data. While traditional stroke risk scores are often used as proxies for predicting ablation outcomes, the researchers demonstrate that XGBoost models—particularly those incorporating specific diagnostic codes—provide a statistically superior, albeit still modest, predictive framework.

TL;DR

Researchers leveraged a massive Medicare claims database ($N=14,521$) to show that machine learning models (XGBoost) significantly outperform traditional clinical risk scores (CHADS₂ and CHA₂DS₂-VASc) in predicting 1-year success after AF ablation. While the overall predictive power remains low, the study highlights that including specific ICD-10 diagnostic codes is crucial for improving model performance.

What Problem Does This Paper Address?

Atrial fibrillation (AF) ablation is a common procedure intended to restore normal heart rhythm, but its success is far from guaranteed—long-term "AF-free" rates range from only 50% to 75%. Currently, clinicians often rely on risk scores like CHADS₂ or CHA₂DS₂-VASc to assess patients. However, these scores were originally designed to predict stroke risk, not ablation success.

There is a significant gap in personalized risk stratification. While granular clinical data (like left atrial size) exists in Electronic Health Records (EHRs), EHR systems are often not interoperable across different hospitals. This paper investigates whether administrative claims data—which is less granular but highly scalable and national in scope—can be used with ML to better identify which patients are likely to benefit from the procedure.

Methods

The researchers compared two supervised learning classifiers against traditional baselines:

  • Models: Logistic Regression (for interpretability) and XGBoost (an ensemble method using gradient-boosted decision trees to capture non-linear relationships).
  • Baselines: CHADS₂ and CHA₂DS₂-VASc risk scores.
  • Feature Sets: The team tested three combinations of features:
  • Demographics only (age, sex, region, industry).
  • Demographics + Comorbidity Indices (Charlson and Elixhauser).
  • Demographics + Comorbidity Indices + 3-digit ICD codes (International Classification of Diseases).
  • Outcome Definition: "Success" was defined as the absence of recurring AF or repeat ablation between 6 and 12 months following the initial procedure.

Datasets

The study utilized the Merative MarketScan® Research Medicare Database, covering claims from January 1, 2013, to December 31, 2020.

  • Total Cohort: 14,521 patients.
  • Subgroups: A subset of 7,646 patients (post-October 2015) was analyzed using ICD-10 codes, which allowed the researchers to distinguish between paroxysmal AF (intermittent) and persistent AF (continuous).

Key Insights and Ideas

  • Traditional Scores are Inadequate: In this context, CHADS₂ and CHA₂DS₂-VASc performed near or even below the level of random chance (AUC ≈ 0.50). This confirms that stroke risk is a poor proxy for rhythm-control success.
  • The "ICD Boost": The inclusion of ICD codes provided the highest predictive lift. This suggests that the specific "medical history footprint" found in billing codes contains latent patterns that traditional weighted indices (like the Charlson Comorbidity Index) miss.
  • Subgroup Sensitivity: ML models performed best for patients with paroxysmal AF (AUC 0.546). This is likely because paroxysmal AF is generally more responsive to ablation, and the models were better at identifying the specific clinical signatures associated with success in this group.
  • Scalability vs. Granularity: The study proves that while claims data lacks the "gold standard" clinical measurements (like echocardiogram results), its sheer volume and national reach make it a viable tool for broad-based clinical decision support.

Key Results

  • Overall Performance: XGBoost achieved an Area Under the Curve (AUC) of 0.525 for the general population, significantly outperforming the baseline scores ($p < 0.001$).
  • Gender Differences: The model was slightly more accurate for males (AUC 0.527) than females (AUC 0.521).
  • Best Performing Model: The model using ICD codes + Demographics + Comorbidity indexes for the paroxysmal AF subgroup reached an AUC of 0.546.
  • Success Rate: Within the study population, the actual success rate of AF ablation was 54.01%.

Limitations and Future Work

  • Modest Predictive Power: While the ML models beat the baselines, an AUC in the 0.52–0.55 range is still considered "weak" for clinical deployment. This highlights the inherent difficulty of predicting ablation outcomes without physiological data (e.g., atrial fibrosis levels or AF duration).
  • Administrative Errors: Claims data is subject to billing and coding errors which can introduce noise into the training set.
  • Population Bias: The dataset was limited to Medicare patients (older adults), meaning the results may not generalize to younger patients with commercial insurance.
  • Future Directions: The authors suggest that future models should attempt to integrate EHR data with claims data to combine the "depth" of clinical measurements with the "breadth" of national longitudinal history.

Summary updated

Abstract

Abstract Background Atrial fibrillation (AF) ablation is an effective treatment for reducing episodes and improving quality of life in patients with AF. However, in some patients there are only modest long-term AF-free rates after AF ablation. There is a need to address the limited benefits some patients experience by developing predictive algorithms to improve AF ablation outcomes. Objective The authors aim to utilize machine learning models on claims data to explore if innovative coding models may lead to better patient outcomes than use of traditional stroke risk score prediction. Methods The Merative MarketScan® Research Medicare data was used to examine claims for AF ablation. To predict 1-year AF-free outcomes after AF ablation, logistic regression and XGBoost models were used. Model predictions were compared with established risk scores CHADS 2 and CHA 2 DS 2 -VASC. These models were also assessed on subgroups of patients with paroxysmal AF, persistent AF, and both AF and atrial flutter from October 2015 onwards. Results The sample included 14,521 patients with claims for AF ablation. XGBoost achieved an area under the receiver operating characteristic curve (AUC) of 0.525, 0.521, and 0.527 for the entire AF ablation population, female, and male, respectively. Within the subgroups, machine learning models performed the best for the paroxysmal AF subgroup using ICD codes, demographic information, and comorbidity indexes, achieving an AUC of 0.546. Conclusion Machine learning models outperformed CHADS 2 and CHA 2 DS 2 -VASC in all AF ablation patient groups (whole population, female, and male). Using patient data for those who had their AF ablation on or after October 2015, machine learning models performed best in all subgroups and the population, indicating that including ICD codes in machine learning models may improve performance.
BibTeX
@misc{liu2024predicting,
  author = {Yijun Liu and Mustapha Oloko-Oba and Kathryn Wood and Michael S. Lloyd and Joyce C. Ho and Vicki Hertzberg},
  title = {{Predicting Atrial Fibrillation Ablation Outcomes: A Machine Learning Approach Leveraging a Large Administrative Claims Database}},
  year = {2024},
  doi = {10.1101/2024.11.16.24317420},
  howpublished = {medRxiv},
}
Figure