Characterizing database granularity using SNOMED-CT hierarchy.

Anna Ostropolets, Christian Reich, Patrick Ryan, Chunhua Weng, Anthony Molinaro, Frank DeFalco, Jitendra Jonnagaddala, Siaw‐Teng Liaw, Hokyun Jeon, Rae Woong Park, Henry M. Spotnitz, Karthik Natarajan, George Argyriou, Kristin Kostka, Robert Miller, Andrew E. Williams, Evan Minty, Jose Posada, George HripcsakPubMed 2020

TL;DR

This paper tackles the challenge of varying levels of detail (granularity) in clinical data from different sources, which can hinder multi-center research. They developed and tested three methods to quantify this granularity using the SNOMED-CT hierarchy, finding that data provenance and country of origin correlate with granularity levels.

What Problem Does This Paper Address?

In healthcare research, especially when combining data from multiple institutions (multi-center studies), a significant hurdle is the heterogeneity of the data. Even when data is standardized into common formats like the OMOP CDM, the underlying clinical concepts used to represent diagnoses, symptoms, and signs can vary greatly in their level of detail. This "granularity" difference means that a concept representing "kidney failure" in one dataset might be very specific (e.g., "Stage 3 chronic kidney disease due to hypertension"), while in another, it might be much broader (e.g., "Renal impairment"). This inconsistency makes it difficult to define patient cohorts accurately and consistently across different data sources, potentially leading to biased results or missed patient populations. The paper aims to address this by providing a way to measure and compare the granularity of these clinical data sources.

Methods

The core of the paper's methodology lies in developing a "granularity score" for each data source. This score is calculated by analyzing how concepts within a data source are represented in the SNOMED-CT hierarchy. They proposed three distinct approaches, all leveraging SNOMED-CT's hierarchical structure:

  1. ICD10-CM Anchor Approach: This method uses three-character ICD10-CM codes (excluding certain categories) as "anchor" points. The idea is that these codes, when mapped to SNOMED-CT, might offer a more stable granularity measure, as ICD10-CM codes are thought to have less variable granularity across different disease areas compared to SNOMED-CT itself.
  2. Broad SNOMED-CT Anchor Approach: This approach uses the very general SNOMED-CT term "Clinical Finding" as the sole anchor. The assumption here is that different diseases are represented at roughly the same level of detail below this broad term across the hierarchy.
  3. Hand-Selected SNOMED-CT Anchor Approach: This method involves selecting a curated set of 22 broad SNOMED-CT terms that represent major groups of conditions. The goal was to manually ensure these anchor concepts were at a similar level of granularity to provide a consistent baseline for comparison.

For each approach, they calculated the "distance" (number of hierarchical steps) from concepts found in a data source to the chosen anchor concept(s). This distance, along with the frequency of concepts at each level, was used to compute a weighted granularity score. The paper also mentions a separate analysis of "vocabulary granularity" to understand how the source coding schemas themselves contribute to the observed data source granularity.

Datasets

The study utilized data from seven data partners, encompassing twenty-two data sources. These sources were a mix of US (14) and non-US (8) locations. The data originated from various sources, including administrative claims, hospital charge data, and electronic health records from different types of healthcare settings (teaching hospitals, primary/secondary practices). The paper notes that detailed descriptions of these sources, including the number of condition records and unique condition concept codes, are available on a GitHub page. No new benchmarks were introduced; rather, existing data sources were analyzed.

Key Insights and Ideas

  • Granularity Matters for Real-World Research: The paper strongly emphasizes that understanding data source granularity is crucial for accurate phenotyping and cohort definition in multi-center observational studies.
  • SNOMED-CT is a Powerful Tool for Granularity Analysis: The hierarchical structure of SNOMED-CT provides a robust framework for quantifying and comparing the level of detail in clinical concepts across different data sources.
  • Provenance and Origin Influence Granularity: There's a clear correlation between where the data comes from (e.g., administrative claims vs. EHRs, US vs. international) and its granularity. This suggests that the why and how data is collected significantly impacts its detail.
  • Consistency Across Methods is Promising: The fact that three different approaches to calculating granularity showed consistent results in classifying data sources into broad categories (low, moderate, high) lends credibility to their findings.
  • Small Data Sources are Tricky: The study highlights that small data sources can exhibit inconsistent granularity scores across different methods, posing a challenge for accurate assessment.

Key Results

  • Three Granularity Levels: The three tested approaches consistently classified the 22 data sources into three broad empirical levels of granularity: high, moderate, and low.
  • Provenance Correlations:
    • Administrative claims and hospital charge data tended to be more granular than EHR data, especially those from primary or specialty practices.
  • Large hospital EHRs also showed high granularity.
  • International data sources were, on average, less granular than US sources.
  • Specific Examples: For chronic kidney disorder, they found that in some data sources (like LPD Australia), a broad concept like "Renal impairment" accounted for a significant portion (23%) of related records, indicating lower granularity and the potential for misclassification if not accounted for.
  • Moderate Granularity Dominates: Most of the analyzed data sources fell into the "moderate granularity" category.
  • High Granularity Sources Identified: AU-ePBRN, MDCR, CUMC, and Ajou University database were identified as having high granularity.
  • Inconsistent Granularity: MIMIC3 was noted as a data source with inconsistent granularity scores across the different approaches.

Limitations and Future Work

  • Limitations:
    • The study did not perform a detailed SNOMED-CT auditing for hierarchy inconsistencies, assuming they wouldn't disproportionately affect all sources.
  • The analysis was limited to "conditions" and did not include procedures or measurements, which could impact overall granularity assessment.
  • The implications of different granularity levels on patient selection and the performance of computable algorithms were not evaluated.
  • Small data sources showed inconsistency in granularity scores across different methods.
  • Future Work:
    • Explore SNOMED-CT's "has-a" relationships in addition to hierarchical links for assessing granularity.
  • Develop methods to compare hierarchy-based and attribute-based granularity approaches.
  • Further characterize factors contributing to data source granularity, such as disentangling coding variances from data transformation aspects.
  • Investigate how granularity impacts patient selection and algorithm performance.

Summary updated

Abstract

Multi-center observational studies require recognition and reconciliation of differences in patient representations arising from underlying populations, disparate coding practices and specifics of data capture. This leads to different granularity or detail of concepts representing the clinical facts. For researchers studying certain populations of interest, it is important to ensure that concepts at the right level are used for the definition of these populations. We studied the granularity of concepts within 22 data sources in the OHDSI network and calculated a composite granularity score for each dataset. Three alternative SNOMED-based approaches for such score showed consistency in classifying data sources into three levels of granularity (low, moderate and high), which correlated with the provenance of data and country of origin. However, they performed unsatisfactorily in ordering data sources within these groups and showed inconsistency for small data sources. Further studies on examining approaches to data source granularity are needed.
BibTeX
@inproceedings{ostropolets2020characterizing,
  author = {Anna Ostropolets and Christian Reich and Patrick Ryan and Chunhua Weng and Anthony Molinaro and Frank DeFalco and Jitendra Jonnagaddala and Siaw‐Teng Liaw and Hokyun Jeon and Rae Woong Park and Henry M. Spotnitz and Karthik Natarajan and George Argyriou and Kristin Kostka and Robert Miller and Andrew E. Williams and Evan Minty and Jose Posada and George Hripcsak},
  title = {{Characterizing database granularity using SNOMED-CT hierarchy.}},
  booktitle = {PubMed},
  year = {2020},
}
Figure