Development and validation of an electronic phenotyping algorithm for chronic kidney disease.

Girish N. Nadkarni, Omri Gottesman, James G. Linneman, Herbert Chase, Richard L. Berg, Samira Farouk, Rajiv Nadukuru, Vaneet Lotay, Ian O. Ellis, George Hripcsak, Peggy Peissig, Chunhua Weng, Erwin P. BöttingerPubMed 2014

TL;DR

This paper introduces an automated algorithm for identifying chronic kidney disease (CKD) cases and controls from electronic medical records (EMRs). This novel approach significantly outperforms traditional ICD-9 code-based methods, offering a more accurate and efficient way to identify patients for research and clinical management.

What Problem Does This Paper Address?

Chronic kidney disease (CKD) is a widespread and serious condition affecting millions, often remaining undiagnosed due to its asymptomatic nature. This lack of early detection hinders timely intervention, increasing the risk of cardiovascular disease and end-stage renal disease requiring dialysis or transplantation. Identifying CKD patients, particularly those with co-existing diabetes and hypertension, is crucial for both improving healthcare quality and enabling research into disease progression and treatment. Traditional methods of identifying these patients, primarily relying on ICD-9 diagnostic codes, are inefficient and inaccurate, leading to missed cases and a significant underdiagnosis rate.

Methods

The researchers developed an automated phenotyping algorithm that leverages multiple data sources within Electronic Medical Records (EMRs). This rule-based algorithm integrates:

  • Diagnostic codes: Specifically ICD-9 codes, used for both inclusion (e.g., codes for CKD, diabetes with renal manifestations, hypertensive kidney disease) and exclusion (e.g., acute kidney failure, HIV infection, sickle cell disease).
  • Laboratory results: Primarily focusing on estimated Glomerular Filtration Rate (eGFR). To address variations in eGFR reporting across different labs, they recalculated eGFR using the CKD-EPI formula. They also implemented logic to exclude acute kidney injury (AKI) by filtering out serum creatinine tests performed on consecutive or same-day intervals.
  • Medication records: Used to identify patients on anti-hypertensive drugs.
  • Textual information from notes: Specifically, physician observation reports were searched for exclusion criteria to further refine case identification.

The algorithm was designed to first identify potential CKD cases and controls, and then further classify them into diabetic CKD (DCKD), hypertensive CKD (HCKD), or diabetic/hypertensive CKD (DHCKD) categories. The development process involved creating detailed flowcharts (Figures 1-4) outlining the logic for case and control identification, including specific criteria for diabetes and hypertension.

Datasets

The study utilized data from three participating institutions within the eMERGE (electronic medical records and genomics) Network:

  • Icahn School of Medicine at Mount Sinai (New York, NY, USA): Used Epic Systems EMR, with over 10 years of ICD-9 data.
  • Marshfield Clinic Research Foundation (Marshfield, Wisconsin, USA): Utilized a comprehensive internally developed EMR, with 75% of participants having over 20 years of medical history.
  • Columbia University Medical Center (New York, NY, USA): Employed a vendor-based and self-developed EMR (Allscripts and iNYP), with over 22 years of structured and unstructured data.

For validation, the algorithm was deployed to randomly select 600 CKD cases and 600 controls from the EMR at the primary site (Mount Sinai). Physician reviewers manually reviewed these records to establish a gold standard. Fifty cases and 50 controls were also reviewed at each of the two secondary sites for validation.

Key Insights and Ideas

  • Multi-modal EMR data is key: The success of the algorithm hinges on combining various data types (codes, labs, text) rather than relying on a single source. This provides a more comprehensive and accurate picture of a patient's health status.
  • Standardizing lab values is critical: The approach of recalculating eGFR using the CKD-EPI formula highlights the challenge of inter-laboratory variability in EMR data and the necessity of robust data standardization techniques.
  • Exclusion criteria are as important as inclusion: Effectively filtering out patients with acute kidney injury or other kidney diseases not related to diabetes or hypertension significantly improves the precision of the CKD case identification.
  • ICD-9 codes are insufficient: The dramatic performance difference between the developed algorithm and traditional ICD-9 code-based screening underscores the limitations of relying solely on diagnostic codes for complex phenotyping.
  • Potential for widespread deployment: The algorithm's successful validation across multiple institutions suggests its replicability and potential for integration into various healthcare systems with EMRs.

Key Results

The developed phenotyping algorithm demonstrated significantly superior performance compared to traditional ICD-9 code-based identification:

  • Primary Site (Mount Sinai):
    • Phenotyping Algorithm: Identified 93.43% of cases and 95.84% of controls.
  • Positive Predictive Value (PPV): 95.95%
    • Negative Predictive Value (NPV): 93.25%
    • ICD-9 Codes Only: Identified only 40.06% of cases and 75.04% of controls.
  • PPV (ICD-9): 62.89%
    • NPV (ICD-9): 54.26%
  • Secondary Sites (Marshfield Clinic and Columbia University Medical Center):
    • PPV: 92%
    • NPV: 100%

Furthermore, the study found that only 18.7% of confirmed CKD cases were referred to a nephrologist during their clinical course, highlighting a potential area for clinical application of the algorithm to improve patient referral pathways.

Limitations and Future Work

  • Exclusion of proteinuria/albuminuria: The algorithm does not directly incorporate urine protein or albuminuria measurements, which are key diagnostic criteria for CKD. This was a deliberate choice due to the inconsistent recording of these values in EMRs. However, the authors acknowledge this limitation.
  • Focus on prevalent CKD: The algorithm was primarily designed to classify co-existing prevalent CKD with T2D and/or HTN, assuming these conditions often precede CKD.
  • Potential for referral outside EMR: The low rate of nephrologist referral observed within the EMR might be an underestimate, as some patients may be referred to specialists outside their primary care EMR system.
  • Future Work: The authors suggest that this algorithm could be used to identify specific patient cohorts for research, track disease progression, and ensure quality control by verifying adherence to standard care guidelines. They also envision its integration into clinical decision support systems to provide real-time alerts to clinicians. The potential for identifying genetic associations and improving patient outcomes through timely diagnosis and intervention is also highlighted.

Summary updated

Abstract

Twenty-six million Americans are estimated to have chronic kidney disease (CKD) with increased risk for cardiovascular disease and end stage renal disease. CKD is frequently undiagnosed and patients are unaware, hampering intervention. A tool for accurate and timely identification of CKD from electronic medical records (EMR) could improve healthcare quality and identify patients for research. As members of eMERGE (electronic medical records and genomics) Network, we developed an automated phenotyping algorithm that can be deployed to identify rapidly diabetic and/or hypertensive CKD cases and controls in health systems with EMRs It uses diagnostic codes, laboratory results, medication and blood pressure records, and textual information culled from notes. Validation statistics demonstrated positive predictive values of 96% and negative predictive values of 93.3. Similar results were obtained on implementation by two independent eMERGE member institutions. The algorithm dramatically outperformed identification by ICD-9-CM codes with 63% positive and 54% negative predictive values, respectively.
BibTeX
@inproceedings{nadkarni2014development,
  author = {Girish N. Nadkarni and Omri Gottesman and James G. Linneman and Herbert Chase and Richard L. Berg and Samira Farouk and Rajiv Nadukuru and Vaneet Lotay and Ian O. Ellis and George Hripcsak and Peggy Peissig and Chunhua Weng and Erwin P. Böttinger},
  title = {{Development and validation of an electronic phenotyping algorithm for chronic kidney disease.}},
  booktitle = {PubMed},
  year = {2014},
}
Figure