Abstract
Retrieving information from structured data tables in a large database may be performed with little to no difficulty, but structured data may not always contain all that is needed to retrieve accurate information compared to narratives from clinical notes. The large volume of clinical notes, however, requires special processing to access the information contained in their unstructured format. In this case study, we present a comparison of two techniques (structured data extraction and natural language processing) and we evaluate their utility in identifying a specific patient cohort from a large clinical database.
Introduction
Identifying a "patient cohort"—a group of patients sharing specific clinical characteristics—is the foundation of clinical trials and retrospective research. Traditionally, researchers rely on structured data: the organized rows and columns of diagnosis codes (ICD-9) and procedure codes. However, structured data is notoriously "noisy." Clinicians often assign a diagnosis code to justify a laboratory test for a suspected condition; even if the test is negative, the code remains. Conversely, many clinical nuances are buried in unstructured data: the free-text narratives, progress notes, and discharge summaries written by medical staff. This study investigates whether automated Natural Language Processing (NLP) can unlock this "hidden" data to build more accurate research cohorts than structured queries alone.
Methods and Datasets
The study utilized the MIMIC-III (Medical Information Mart for Intensive Care) database, a massive dataset containing de-identified data from over 58,000 hospital admissions. The researchers compared two distinct approaches to identify a cohort of diabetic patients who underwent hemodialysis:
- Structured Data Extraction: Using SQL queries to search for specific ICD-9 diagnosis codes (e.g., 250.xx for diabetes) and procedure codes (e.g., 39.95 for hemodialysis) within database tables.
- Clinical NLP: Using cTAKES (clinical Text Analysis and Knowledge Extraction System). This method leveraged the UMLS (Unified Medical Language System) to identify synonyms and acronyms (e.g., "DM" for diabetes or "HD" for hemodialysis) and employed negation detection to ignore phrases like "patient denies diabetes."
To evaluate these methods, the authors manually reviewed clinical notes to create a "validation database" of 1,879 patients to serve as the ground truth.
Key Results and Insights
The NLP method significantly outperformed the structured method in both precision and volume. While both methods were highly precise, the structured method suffered from a significant "missingness" problem, failing to identify hundreds of patients clearly documented in the clinical notes.
- Precision: The NLP method achieved 99.2% precision, compared to 98.2% for the structured method.
- Patient Volume: The NLP method identified 1,666 true positive patients, whereas the structured SQL method found only 1,013—representing a nearly 65% increase in cohort size.
- Error Reduction: NLP identified fewer False Positives. In structured data, many errors arose from patients coded for diabetes who actually had unrelated conditions like "diabetes insipidus" or gestational diabetes, which the NLP heuristics successfully filtered out.
The "Context" Advantage
Structured codes are often static snapshots at the time of discharge. In contrast, NLP can track the evolution of a patient's status through daily progress notes. Clinicians frequently use acronyms like "DM" or "HD" in a busy ICU environment; standard keyword searches miss these, but NLP tools mapped to medical ontologies catch them effortlessly. Furthermore, the ability to recognize negation (e.g., "ruled out") was vital for maintaining high precision.
Discussion and Future Work
While the results are promising, the study has limitations. The validation was performed manually, which is labor-intensive, and the findings are based on a single source (MIMIC-III). Additionally, the study did not integrate other structured sources like laboratory results or medication records, which could further refine identification.
Future work suggests applying these NLP techniques to "time-series" analysis—tracking how a patient's condition changes hour-by-hour in the ICU by mining the continuous flow of clinical narratives.
Summary updated