Sepsis-3 criteria in AmsterdamUMCdb: open-source code implementation

Tom Edinburgh, Stephen J. Eglen, Patrick Thoral, Paul Elbers, Ari ErcoleZenodo 2022 · DOI

TL;DR

This paper addresses the challenge of consistently identifying sepsis patients in large ICU datasets, which is crucial for research but hindered by data standardization issues. The authors provide an open-source Python implementation to operationalize the Sepsis-3 definition on the AmsterdamUMCdb, enabling reproducible research and facilitating comparative studies across different critical care settings.

What Problem Does This Paper Address?

Sepsis is a life-threatening condition with high mortality, making its accurate identification and study in intensive care units (ICUs) a priority. Researchers rely on large, de-identified ICU databases for observational studies. However, a major hurdle is the lack of standardized data structures across different Electronic Health Record (EHR) systems and database designs. This variability makes it difficult to reliably select cohorts of patients, such as those with sepsis, which is essential for reproducible research. The Sepsis-3 definition, a consensus guideline, aims to standardize sepsis identification, but its practical implementation in diverse data environments remains challenging. This paper tackles this by providing a concrete, open-source solution for applying the Sepsis-3 criteria to a specific, large European ICU database.

Methods

The core of this work is the development of a Python script that operationalizes the Sepsis-3 definition on the AmsterdamUMCdb. The script calculates daily Sequential Organ Failure Assessment (SOFA) scores, identifies antibiotic escalation events, and flags sepsis and septic shock episodes.

Key methodological aspects include:

  • SOFA Score Calculation: The script extends the standard 24-hour SOFA calculation to cover multiple "days" (24-hour periods) post-admission, allowing for a more comprehensive assessment. Missing SOFA components are handled by assuming a score of zero, but admissions with more than three missing components are excluded from sepsis identification for that day.
  • Infection Identification: Infection is inferred through antibiotic escalation, defined by an increase in the rank of administered antibiotics. This approach acknowledges the limitation that it relies on clinical judgment rather than confirmed infection. The script also accounts for specific local practices, such as the routine use of cefotaxime in the Amsterdam UMC, by disregarding it for the first four days to avoid misclassifying prophylactic use as infection-driven escalation.
  • Sepsis and Septic Shock Definition: A sepsis episode is defined as an acute increase in the total SOFA score of at least two points within a 24-hour period, accompanied by antibiotic escalation. Septic shock is further defined by a cardiovascular SOFA score of at least 3 (indicating vasopressor use) and a lactate level greater than 2 mmol/L.
  • Data Handling: The script works directly with comma-separated value (CSV) files to maintain database agnosticism.

Datasets

The primary dataset used in this study is the Amsterdam University Medical Centers Database (AmsterdamUMCdb). This is a large, freely accessible European ICU database containing data from approximately 20,109 critically ill patients admitted between 2003 and 2016. The database is described as a "data lake" structure with seven CSV tables.

The paper does not introduce a new benchmark dataset but rather provides an implementation for an existing one.

Key Insights and Ideas

  • Bridging the Gap Between Definition and Data: The paper highlights the critical need for practical, implementable tools to translate clinical definitions like Sepsis-3 into actionable code for large-scale data analysis.
  • The Challenge of EHR Data Standardization: The authors underscore that variations in EHR systems and data collection practices are a significant barrier to reproducible research in critical care. Their solution aims to mitigate this by providing a standardized code.
  • Inferring Infection from Antibiotic Use: The reliance on antibiotic escalation as a proxy for infection is a pragmatic approach given the limitations of EHR data. However, the authors are transparent about this being a limitation, as it's not a direct confirmation of infection.
  • Importance of Context-Specific Implementation: The explicit handling of local antibiotic protocols (e.g., cefotaxime use) demonstrates the necessity of tailoring sepsis identification algorithms to the specific context of the data source.
  • Reproducibility as a Cornerstone: By releasing their code as open-source, the authors champion the principles of reproducible research, allowing other critical care researchers to adopt, adapt, and build upon their work.

Key Results

The paper presents a confusion matrix (Table 1) comparing their implemented Sepsis-3 definition against the existing AmsterdamUMCdb sepsis definition for both unique first admissions and all admissions.

  • For unique first admissions, the Sepsis-3 implementation achieved a sensitivity of 0.33 and a specificity of 0.94.
  • For all admissions, the sensitivity was 0.32 and the specificity was 0.94.

These results indicate that the Sepsis-3 definition, as operationalized by the authors, identifies a subset of patients identified by the existing criteria, with high specificity but lower sensitivity. The authors note that the sensitivity of their current criteria compared to Sepsis-3 in the first 24 hours is poor.

The output of the script includes CSV files containing daily SOFA scores and binary indicators for antibiotic escalation, prophylaxis, infection, sepsis episodes, and septic shock.

Limitations and Future Work

Limitations:

  • Inference of Infection: The definition of infection relies on antibiotic escalation, which is an indirect measure and may not always accurately reflect a true infection. This is acknowledged as a limitation.
  • Sensitivity of Sepsis-3 Implementation: The authors note that their implementation shows poor sensitivity for identifying sepsis within the first 24 hours compared to existing criteria.
  • Database Specificity: While the code is designed to be database-agnostic by using CSVs, the specific implementation is tailored to the AmsterdamUMCdb structure.

Future Work:

  • The authors' work inherently supports future research by providing a standardized tool. Further research could involve applying this code to other ICU databases to assess the generalizability of the Sepsis-3 implementation and to conduct comparative studies.
  • Investigating methods to improve the accuracy of infection identification beyond antibiotic escalation would be a valuable next step.
  • Exploring ways to enhance the sensitivity of the Sepsis-3 implementation, particularly in the early hours of admission, is also a potential area for future development.

Summary updated

Abstract

Sepsis is a major healthcare problem with substantial mortality and a common reason for admission to the intensive care unit (ICU). For this reason, the management of sepsis is an important area of ICU research. A number of large-scale, freely-accessible ICU databases are available for observational research and the robust identification of septic patients in such data sets is crucial for research purposes, particularly for comparative studies between critical care sub-populations which may vary around the world. However, data structures are poorly standardised due to inevitable variances in clinical electronic health record system vendor and implementation as well as research database design choices. Robust and well-documented cohort selection (such as patients with sepsis) is crucial for reproducible research. In this work, we operationalise the Sepsis-3 definition on the AmsterdamUMCdb, a recently published large European ICU database, publishing open-access code for wider use by critical care researchers. This is the Author Accepted Manuscript, accepted after peer review by GigaByte.
BibTeX
@inproceedings{edinburgh2022sepsis,
  author = {Tom Edinburgh and Stephen J. Eglen and Patrick Thoral and Paul Elbers and Ari Ercole},
  title = {{Sepsis-3 criteria in AmsterdamUMCdb: open-source code implementation}},
  booktitle = {Zenodo},
  year = {2022},
  doi = {10.17863/cam.82518},
}
Figure