UK phenomics platform for developing and validating electronic health record phenotypes: CALIBER

Spiros C. Denaxas, Arturo Gonzalez-Izquierdo, Kenan Direk, Natalie K. Fitzpatrick, Ghazaleh Fatemifar, Amitava Banerjee, Richard J. B. Dobson, Laurence J. Howe, Valerie Kuan, R. Tom Lumbers, Laura Pasea, Riyaz S. Patel, Anoop D. Shah, Aroon D. Hingorani, Cathie Sudlow, Harry HemingwayJ. Am. Medical Informatics Assoc 2019 · DOI

TL;DR

This paper introduces CALIBER, a comprehensive platform for creating, validating, and sharing reproducible "phenotypes" (measurable characteristics) from UK electronic health records (EHRs). By linking diverse data sources and employing a rigorous validation framework, CALIBER aims to unlock the potential of EHRs for large-scale health research, addressing the fragmentation and variability that has historically hindered their use.

What Problem Does This Paper Address?

Electronic health records (EHRs) are a treasure trove of patient information, but extracting meaningful insights for research is a significant challenge. The data is often unstructured, fragmented across different systems (primary care, hospitals, mortality registries), and collected for clinical care rather than research. This variability in data format, coding systems, and completeness makes it difficult to consistently identify and define specific patient characteristics (phenotypes) needed for studies, especially for complex diseases. While individual research groups have developed methods, there's been a lack of a standardized, reproducible, and shareable approach in the UK, hindering collaborative and large-scale research.

Methods

CALIBER tackles this by implementing a rule-based phenotyping framework that integrates data from multiple UK EHR sources. The core of their approach involves:

  • Iterative Algorithm Development: Phenotyping algorithms are developed collaboratively by a multidisciplinary team (clinicians, epidemiologists, computer scientists) and refined through multiple iterations based on data exploration and clinical context.
  • Cross-Source Data Integration: Algorithms are designed to leverage data from primary care (CPRD), hospital admissions (HES), and mortality registries (ONS), reconciling differences in coding terminologies (Read, ICD-10, etc.) and data structures.
  • Systematic Validation Framework: A key innovation is the use of up to six independent approaches to validate each phenotype. These include:
  • Cross-EHR Concordance: Assessing how well a phenotype is captured across different data sources.
  • Case-Note Review: Expert review of individual patient records to determine the "gold standard" accuracy of the algorithm.
  • Etiology: Checking if known risk factors for a disease are consistently identified in the EHR data.
  • Prognosis: Evaluating if the EHR-derived phenotypes predict known disease outcomes.
  • Genetic Associations: Replicating known genetic links to diseases using the phenotypes.
  • External Populations: Testing the phenotype's validity in data from other countries.
  • Open-Access Phenotype Portal: All developed phenotypes, including their implementation logic and validation evidence, are made publicly available through the CALIBER Portal, promoting transparency and reusability.

Datasets

The CALIBER platform utilizes several large-scale UK national EHR data sources:

  • Clinical Practice Research Datalink (CPRD): A primary care EHR database containing longitudinal data on diagnoses, symptoms, medications, and more.
  • Hospital Episode Statistics (HES): Administrative data from hospital admissions, including diagnoses (ICD-10) and surgical procedures.
  • Office for National Statistics (ONS): Mortality data, including cause-specific mortality.
  • Myocardial Ischaemia National Audit Project (MINAP): A national registry for acute coronary syndrome events, used as a reference for AMI validation.

These datasets are linked deterministically using NHS numbers and other demographic information, covering a sample of 15 million individuals.

Key Insights and Ideas

  • The Power of Linkage: Linking primary care, hospital, and mortality data is crucial for a comprehensive understanding of disease trajectories, as no single source captures the full picture.
  • Validation is Paramount: Simply creating an algorithm isn't enough; rigorous, multi-faceted validation is essential to build confidence in EHR-derived phenotypes for research.
  • Reproducibility is Achievable: By meticulously documenting algorithms, logic, and validation evidence, CALIBER demonstrates that reproducible phenotyping from complex EHR data is possible.
  • The "Phenotype" is More Than Codes: A true phenotype definition includes not just the clinical codes but also the precise logic for their use and robust evidence of validity.
  • UK's Unique Advantage: The UK's integrated healthcare system and extensive national data resources provide a unique opportunity for developing such comprehensive phenotyping platforms.

Key Results

CALIBER has successfully developed and curated 51 validated phenotyping algorithms for diseases, syndromes, biomarkers, and lifestyle risk factors. These phenotypes have been published on the CALIBER Portal and have already been used by 40 national and international research groups in 60 peer-reviewed publications. The validation approaches have provided strong evidence for the accuracy and utility of these phenotypes, with examples showing high positive predictive values (e.g., 92.2% for AMI in primary care) and consistent associations with known risk factors and prognostic outcomes.

Limitations and Future Work

The authors acknowledge several limitations:

  • Labor-Intensive Process: Developing and validating phenotypes is a time-consuming and labor-intensive process, often requiring numerous iterations.
  • Limited Case-Note Review: While case-note review is considered a gold standard, it was only performed in a single instance due to data access constraints, limiting its broader application.
  • Site-Level Variability: The use of aggregated data means the impact of site-specific data capture and quality variations could not be fully assessed.

Future work will focus on:

  • Scaling Up: Expanding the number of phenotypes to cover a wider range of diseases and characteristics.
  • Automation and Machine Learning: Exploring high-throughput phenotyping methods, including machine learning and natural language processing, to accelerate the process.
  • Interoperability: Developing computable phenotypes that can be integrated into the healthcare system and used for clinical decision support.
  • Cross-Biobank Analyses: Enabling more complex research by facilitating the use of CALIBER phenotypes across different data sources and international collaborations.

Summary updated

BibTeX
@article{denaxas2019uk,
  author = {Spiros C. Denaxas and Arturo Gonzalez-Izquierdo and Kenan Direk and Natalie K. Fitzpatrick and Ghazaleh Fatemifar and Amitava Banerjee and Richard J. B. Dobson and Laurence J. Howe and Valerie Kuan and R. Tom Lumbers and Laura Pasea and Riyaz S. Patel and Anoop D. Shah and Aroon D. Hingorani and Cathie Sudlow and Harry Hemingway},
  title = {{UK phenomics platform for developing and validating electronic health record phenotypes: CALIBER}},
  journal = {J. Am. Medical Informatics Assoc.},
  year = {2019},
  doi = {10.1093/jamia/ocz105},
}
Figure