Normalization of relative and incomplete temporal expressions in clinical narratives

Weiyi Sun, Anna Rumshisky, Özlem Uzuner

TL;DR

Normalizing relative time expressions (like "two days later") in clinical notes is notoriously difficult because standard NLP rules built for news articles don't apply to healthcare timelines. This paper introduces a machine learning approach using two multi-label classifiers to identify temporal "anchor points" and "anchor relations," significantly outperforming existing rule-based systems on clinical datasets.

What Problem Does This Paper Address?

To build accurate patient timelines from clinical notes, Natural Language Processing (NLP) systems must extract and normalize Temporal Expressions (TIMEXes) into a standard format (like ISO8601). While absolute dates ("Nov 29, 2013") are straightforward, Relative and Incomplete TIMEXes (RI-TIMEXes)—such as "two days after surgery" or "at 6 AM"—require contextual clues to resolve.

Existing systems usually assume these relative expressions anchor to the Document Creation Time (DCT). This heuristic works incredibly well for news articles, but fails miserably in clinical discharge summaries. In clinical narratives, the timeline constantly jumps between admission dates, discharge dates, and various procedural events, making standard out-of-the-box temporal NLP tools highly inaccurate.

Methods

The authors reframe RI-TIMEX normalization as a pair of multi-label classification problems rather than relying purely on rigid, hand-crafted rules. Their pipeline consists of:

  1. Anchor Point Classifier: A Support Vector Machine (SVM) model that links an RI-TIMEX to a specific reference point. It classifies the anchor as either the Admission Date, Discharge Date, a Previous TIMEX, or a Previous Absolute TIMEX.
  2. Anchor Relation Classifier: Another SVM that determines the temporal relationship between the RI-TIMEX and its anchor (categorized as "before", "after", or "equal/during").
  3. Rule-Based Parser & Integrator: A module that parses the text span (e.g., converting the string "two days" to a numeric value of 2) and applies the predicted relation to the anchor point to calculate the final standardized date/time.

The researchers experimented with various feature sets to train these classifiers, including bag-of-words, normalized numbers, clinical event tags, and verb tenses.

Datasets

  • 2012 i2b2 Temporal Relation Challenge Corpus: The primary dataset used for training and evaluation, consisting of 310 clinical discharge summaries.
  • TimeBank: A newswire corpus (183 articles) used purely for comparative annotation analysis.
  • WikiWars: A historical narrative corpus (22 Wikipedia articles) also used for comparative analysis.

Key Insights and Ideas

  • Domain differences are stark: The authors' annotation study revealed exactly why standard tools fail on medical text. In newswire text, 96% of relative times anchor to the Document Creation Time. In clinical texts, anchors are highly distributed (53% to admission date, 37% to a previously mentioned time, 16% to discharge date).
  • Verb tense is useless in clinical notes: The authors hypothesized that verb tense would help predict anchor relations (e.g., past tense implies "before"). However, because discharge summaries are almost entirely written in the past tense, this feature added no predictive value and was dropped.
  • Number normalization is a double-edged sword: Converting all numbers to a generic <NUM> token improved the model's ability to find the anchor point, but it destroyed its ability to find the anchor relation. For example, "day of life #1" and "day of life #2" anchor to the same birth event, but the specific number dictates whether the relation is "equal/during" or "after."

Key Results

  • The system achieved 74.68% accuracy for anchor point classification and 87.71% accuracy for anchor relation classification.
  • For end-to-end normalization, the system achieved 57.2% accuracy under strict matching criteria.
  • Because clinical notes are often ambiguous (e.g., does "post-operative day 1" mean the day of surgery or the day after?), the authors used a relaxed evaluation metric allowing a +/- 1 day deviation. Under this relaxed metric, normalization accuracy jumped to 82.09%.
  • Compared to the top-performing systems from the original 2012 i2b2 challenge, this approach achieved a statistically significant improvement, boosting the relaxed F1-score from 45.42 to 53.77.

Limitations and Future Work

Limitations: The supervised machine learning approach relies heavily on datasets that have explicit, high-quality annotations for anchor points and relations—which are expensive and rare to produce. Furthermore, the system was only evaluated on the i2b2 corpus (discharge summaries), so its generalizability to other types of clinical notes (like daily progress notes or outpatient clinic notes) remains unproven.

Future Work: The authors suggest that because the system's architecture is relatively simple and modular, it could be readily extended and adapted to other corpora or entirely different domains, provided the right training data is available.

Summary updated

Abstract

We analyze the RI-TIMEXes in temporally annotated corpora and propose two hypotheses regarding the normalization of RI-TIMEXes in the clinical narrative domain: the anchor point hypothesis and the anchor relation hypothesis. We annotate the RI-TIMEXes in three corpora to study the characteristics of RI-TMEXes in different domains. This informed the design of our RI-TIMEX normalization system for the clinical domain, which consists of an anchor point classifier, an anchor relation classifier and a rule-based RI-TIMEX text span parser. We experiment with different feature sets and perform error analysis for each system component. The annotation confirmed the hypotheses that we can simplify the RI-TIMEXes normalization task using two multi-label classifiers. Our system achieves anchor point classification, anchor relation classification and rule-based parsing accuracy of 74.68%, 87.71% and 57.2% (82.09% under relaxed matching criteria) respectively on the held-out test set of the 2012 i2b2 temporal relation challenge. Experiments with feature sets reveals some interesting findings such as the verbal tense feature does not inform the anchor relation classification in clinical narratives as much as the tokens near the RI-TIMEX. Error analysis shows that underrepresented anchor point and anchor relation classes are difficult to detect. We formulate the RI-TIMEX normalization problem as a pair of multi-label classification problems. Considering only the RI-TIMEX extraction and normalization, the system achieves statistically significant improvement over the RI-TIMEX results of the best systems in the 2012 i2b2 challenge.
BibTeX
@article{sun2015normalization,
  author = {Weiyi Sun and Anna Rumshisky and Özlem Uzuner},
  title = {{Normalization of relative and incomplete temporal expressions in clinical narratives}},
  journal = {J. Am. Medical Informatics Assoc.},
  year = {2015},
  url = {https://openreview.net/forum?id=9fOlPccrq1S},
}

Loading related content...

Figure