A Pattern-Based Approach to Recognizing Time Expressions

Wentao Ding, Guanji Gao, Linfeng Shi, Yuzhong QuAAAI 2019 · DOI

Introduction

Recognizing time expressions (TER)—such as "three days ago" or "late in 2018"—is a fundamental task in natural language processing. Historically, practitioners have faced a trade-off: the high performance of "black-box" deep learning models versus the transparency and debuggability of hand-crafted rules. In research presented at AAAI-19, Nanjing University researchers introduced PTime, a system that achieves state-of-the-art performance while remaining entirely interpretable by treating pattern selection as a mathematical optimization problem.

The Challenge of Time Expressions

Time expressions are context-dependent and linguistically diverse. A word like "in" can be temporal ("late in 2018") or spatial ("in the office"). Existing tools typically fall into two categories: rule-based semantic parsers, which are easy to understand but often suffer from low recall, and machine learning models (like CRFs or Deep Learning), which are accurate but offer no explanation for their predictions. PTime addresses the need for a system that is interpretable, adjustable via a "mistake budget," and competitive with modern black-box architectures.

Methodology

PTime operates through a three-stage pipeline designed to automate pattern discovery and selection.

The PTime system architecture, illustrating the transition from training documents to pattern-based extraction.
  1. Abstraction via Token Types: Text is converted into sequences of 32 fine-grained types (e.g., DEFINITE_DET, TIME_UNIT, MODIFIER). To handle domain-specific language, PTime dynamically creates "literal" types for words that do not fit predefined categories.
  2. Pattern Generation: The system extracts all sequences of token types corresponding to known time expressions in the training data.
  3. Extended Budgeted Maximum Coverage (EBMC): To filter out overly general patterns that cause false positives (e.g., a pattern for "three quarters of an hour" incorrectly flagging "three quarters of the cake"), PTime uses the EBMC optimization model. It selects a subset of patterns that maximizes the "Gain" (correctly identified tokens) while keeping the "Cost" (mistakes) under a budget $B$, controlled by a parameter $\rho$.

Experimental Evaluation

The authors evaluated PTime on three standard benchmarks: TempEval-3 (general temporal extraction), WikiWars (complex historical narratives), and Tweets (informal social media text).

Key Results

PTime demonstrated that optimized pattern matching can outperform complex black-box models. On the WikiWars and Tweets datasets, PTime achieved the highest F1 scores reported to date.

Performance comparison on the WikiWars dataset, showing PTime achieving superior F1 scores in both strict and relaxed matching.

As shown in the WikiWars results, PTime improved the strict match F1 score by over 3% compared to previous state-of-the-art models like ClearTK and UWTime. On the Tweets dataset, it similarly outperformed models like TOMN and SynTime. While it placed 3rd on TempEval-3, it remained highly competitive while offering significantly better interpretability.

Discussion and Insights

A core contribution of PTime is the "knob" provided by the $\rho$ parameter. In industrial settings where false positives are costly, engineers can tune $\rho$ to prioritize precision. Furthermore, because the output consists of human-readable patterns, developers can inspect and prune specific rules that cause errors. By allowing untyped tokens to remain as literals, the system avoids the "closed-world" limitations of traditional rule-based systems, enabling it to learn new time units (like "hr") dynamically.

Limitations and Future Work

Despite its performance, PTime currently lacks broad contextual awareness, relying primarily on the string itself to identify expressions. It may struggle with novel formats like "3:07:35" if they were absent from training data, even if the surrounding text implies a temporal meaning. Future research may explore semi-automatic token type design and integration with advanced normalization techniques to better resolve the specific values of identified time expressions.

Summary updated

BibTeX
@inproceedings{ding2019pattern,
  author = {Wentao Ding and Guanji Gao and Linfeng Shi and Yuzhong Qu},
  title = {{A Pattern-Based Approach to Recognizing Time Expressions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33016335},
}
Figure