TL;DR
This paper introduces AllergyMap, the first publicly available corpus for normalizing free-text allergy mentions from electronic health records (EHRs) to standard terminologies. This resource is crucial for improving patient safety and interoperability during EHR migrations by enabling better allergy data extraction and analysis.
What Problem Does This Paper Address?
Accurately documenting patient allergies is critical for safe and effective healthcare, yet it's surprisingly challenging. Allergy mentions in EHRs are often free-text, leading to a wide variety of terms, misspellings, and even non-allergy information. This ambiguity makes it difficult for automated systems to reliably identify and process allergies, potentially leading to incorrect prescribing, adverse reactions, and increased healthcare costs. Furthermore, migrating this messy free-text data to new EHR systems is a significant hurdle. Existing NLP tools and corpora for allergy normalization are scarce, proprietary, or not publicly available, hindering research and development in this area.
Methods
The authors developed a systematic, iterative process for mapping free-text allergy entries to standard terminologies. This involved:
- Data Extraction: Extracting active, free-text allergy entries from an existing inpatient EHR system.
- Initial Culling: Removing entries that were ambiguous, irrelevant (e.g., patient weight, pregnancy status), or too vague (e.g., "unspecified topicals").
- Iterative Clinical Review: An interdisciplinary team of clinicians, informaticians, and IT professionals reviewed and mapped the remaining entries. This involved manual review, mapping to standard terminologies (like RxNorm and NDF-RT), and identifying terms that couldn't be mapped but were clinically significant enough to be retained as text annotations.
- Final Review and UMLS Mapping: Physicians with informatics expertise performed a final review, adjudicating mapping decisions and assigning a Unified Medical Language System (UMLS) Concept Unique Identifier (CUI) to each mapped term.
- Inter-Annotator Agreement: A subset of 100 entries was independently annotated by two individuals to assess the reliability of the mapping process, using F-measure as the evaluation metric.
Datasets
The primary dataset used was free-text allergy entries extracted from the existing inpatient EHR system at Northwestern Memorial Hospital, spanning from June 27, 2002, to November 25, 2015. After initial processing, 8,153 unique entries remained, and after duplicate removal, 2,549 unique entries were used for the next phase of review.
The paper introduces AllergyMap, a newly created corpus of normalized free-text allergy mentions. This corpus is made freely available and serves as a benchmark for future allergy mention normalization research. It differs from existing resources by being publicly available and derived from a comprehensive manual mapping and cleaning process.
Key Insights and Ideas
- The "Messiness" of Real-World Data: Free-text allergy fields are a dumping ground for various types of information, not just allergies. Clinicians use them for everything from drug sensitivities to pregnancy status and fluid restrictions, making simple keyword matching insufficient.
- The Importance of Manual Curation: Despite the desire for automation, manual review and mapping are essential for creating a high-quality corpus. The authors found that even with NLP tools, manual validation is necessary for accuracy, especially with the complexity and variety of allergy mentions.
- The "Absence of Allergy" is Important Information: A significant finding was that many free-text entries described the absence of allergies or intolerances. This information, while not a direct allergy mention, is crucial for patient safety and should be preserved, often as text annotations.
- Proprietary Vocabularies and Heterogeneity are Major Hurdles: The lack of comprehensive, standardized allergy terminologies and the use of proprietary systems significantly complicate normalization efforts. Allergens span a vast range, from specific drugs to broad categories like "shellfish" or "environmental allergens."
Key Results
- After initial processing and de-duplication, 2,549 unique free-text allergy entries were mapped.
- Of these, 2,237 entries were successfully mapped to standard terminologies.
- 312 entries were deemed unmappable, with 170 of these recommended for retention as text annotations in the new EHR system.
- The inter-annotator agreement, measured by F-measure, was 0.83, indicating a reliable and reproducible mapping process.
- The AllergyMap corpus is now freely available for public use.
Limitations and Future Work
- Data Source Limitations: The allergy data was sourced from a single university hospital's inpatient EHR, and only text strings, status, and reaction type were available, lacking crucial context like hypersensitivity type or severity.
- Mapping Non-Drug Allergens: Non-drug allergens were mapped to NDF-RT, which was required by the target EHR but may not align with current interoperability standards.
- Reliance on Manual Effort: The current process heavily relies on manual review, which is time-consuming and requires specialized expertise.
- Future Work: The authors plan to explore methods for increasing automation in the mapping process while optimizing the workflow and usability of manual mapping. They also aim to use the AllergyMap corpus to train and develop NLP allergy normalization software. They welcome stakeholder contributions to make the corpus more robust and applicable to various use cases.
Summary updated