A Comprehensive Typing System for Information Extraction from Clinical Narratives

J. Harry Caufield, Yichao Zhou, Yunsheng Bai, David A. Liem, Anders O. Garlid, Kai-Wei Chang, Yizhou Sun, Peipei Ping, Wei WangmedRxiv 2019 · DOI

Abstract

We have developed ACROBAT (Annotation for Case Reports using Open Biomedical Annotation Terms), a typing system for detailed information extraction from clinical text. This resource supports detailed identification and categorization of entities, events, and relations within clinical text documents, including clinical case reports (CCRs) and the free-text components of electronic health records. Using ACROBAT and the text of 200 CCRs, we annotated a wide variety of real-world clinical disease presentations. The resulting dataset, MACCROBAT2018, is a rich collection of annotated clinical language appropriate for training biomedical natural language processing systems.

Introduction

Clinical narratives, such as clinical case reports (CCRs) and electronic health records (EHRs), are a goldmine of medical knowledge. However, extracting this information automatically remains a significant challenge for modern informatics. Unlike structured data (e.g., a lab result in a database table), these narratives are written in natural language where context is everything.

Existing medical coding systems like ICD-10 or SNOMED CT are often too rigid or hierarchical to capture the nuances of clinical text. For instance, a report might mention "microscopic examination of a biopsy revealed a typical LCH." To assign a proper code, a system needs to know if the disease is "unifocal" or "disseminated"—details that might not be explicitly stated in that specific sentence. Furthermore, general biomedical NLP tools often struggle with modifiers; the word "elongated" means something very different when describing a heart valve versus a bone in the skull. There is a clear need for a system that captures the semantics of usage rather than just mapping words to a static dictionary.

The ACROBAT Typing System

The authors developed ACROBAT (Annotation for Case Reports using Open Biomedical Annotation Terms), a "typing system" designed to bridge the gap between messy text and structured data. Key features of the method include a clear distinction between "Events" (things that happen in time, like a surgery or a symptom) and "Entities" (modifiers or descriptions, like the color of a lesion or the dosage of a drug).

Definitions of core entity and event types in the ACROBAT system.

The system tracks granular properties such as Polarity (is the symptom present, absent, or uncertain?) and Trend (is the condition improving, worsening, or staying the same?). Instead of just identifying words, ACROBAT maps how they relate using a "MODIFY" relation for general descriptions and specific temporal relations (BEFORE, AFTER, OVERLAP) to reconstruct a patient's timeline. A term like "C-reactive protein" is labeled as a "Diagnostic procedure" if the context implies a test was done, even if the word "test" is missing.

Example of ACROBAT annotations in a clinical case report snippet.

The MACCROBAT2018 Dataset

The primary output of this work is the MACCROBAT2018 dataset, which consists of 200 open-access clinical case reports from PubMed Central. This corpus contains 3,652 sentences and 59,164 total annotations, providing a high density of information for training biomedical natural language processing (BioIE) systems.

Key statistics from the dataset include:

  • Diagnostic Procedures: 9,135 instances (the most frequent category).
  • Signs/Symptoms: 6,718 instances.
  • Lab Values: 3,565 instances.
  • Relations: Over 11,000 "MODIFY" relations and nearly 4,400 temporal relations.

Key Principles and Insights

  • The "Smallest Span" Rule: To ensure high-quality data, the authors insist on labeling the smallest meaningful span. In "massive heart attack," the event is "heart attack," while "massive" is a separate "Severity" entity. This allows models to learn the nuances of medical modifiers.
  • Knowledgebase Linkage: ACROBAT includes specific types like Gene_or_protein and Diagnostic_standard to help future models link clinical text directly to external biological databases (like UniProt).
  • Temporal Logic: By requiring all events to be linked via temporal relations, the system forces the annotation to reflect the actual progression of a disease, which is critical for longitudinal patient care.
  • Semantics over Ontology: The authors argue that reflecting how terminology is actually used by clinicians is more practical for building real-world AI than trying to force text into a pre-defined, perfect hierarchy.

Conclusion and Future Work

While MACCROBAT2018 is a strong start, the authors acknowledge limitations in corpus size and language diversity (currently limited to English). Future releases are expected to add more documents and expand the variety of clinical scenarios covered. For now, ACROBAT provides a robust framework for anyone looking to turn "doctor-speak" into actionable data.

Summary updated

Abstract

Abstract We have developed ACROBAT (Annotation for Case Reports using Open Biomedical Annotation Terms), a typing system for detailed information extraction from clinical text. This resource supports detailed identification and categorization of entities, events, and relations within clinical text documents, including clincal case reports (CCRs) and the free-text components of electronic health records. Using ACROBAT and the text of 200 CCRs, we annotated a wide variety of real-world clinical disease presentations. The resulting dataset, MACCROBAT2018, is a rich collection of annotated clinical language appropriate for training biomedical natural language processing systems.
BibTeX
@misc{caufield2019comprehensive,
  author = {J. Harry Caufield and Yichao Zhou and Yunsheng Bai and David A. Liem and Anders O. Garlid and Kai-Wei Chang and Yizhou Sun and Peipei Ping and Wei Wang},
  title = {{A Comprehensive Typing System for Information Extraction from Clinical Narratives}},
  year = {2019},
  doi = {10.1101/19009118},
  howpublished = {medRxiv},
}
Figure