Fine-Grained Temporal Relation Extraction

Siddharth Vashishtha, Benjamin Van Durme, Aaron Steven White

Presented at ACL 2019, this research marks a significant shift in how Natural Language Processing (NLP) handles temporal information. Rather than treating temporal relations as a discrete classification task (e.g., "Event A happened before Event B"), the authors propose a framework that maps events onto a continuous, real-valued timeline.

Addressing the Limitations of Categorical Time

In natural language, time is often implicit. For example, in the sentence, "The boy broke the window and ran away," a human reader instinctively understands that the "breaking" was nearly instantaneous, while the "running" occupied a longer duration.

Traditional NLP frameworks, such as TimeML, struggle with these nuances because they rely on categorical labels like BEFORE, AFTER, or OVERLAP. This approach suffers from two primary flaws:

  1. Implicit Information Loss: It typically only captures duration when explicitly stated (e.g., "for two hours").
  2. Relational Sparsity: Categorical labels are often "brittle" and do not easily support the construction of a unified document-level timeline, particularly when relations between distant events are not explicitly mentioned.

The authors argue that duration must be a first-class citizen to truly reconstruct the temporal structure of a narrative.

A conceptual timeline where events are mapped to continuous intervals rather than discrete categories.

The UDS-T Framework and Dataset

The core contribution of this work is the UDS-T (Universal Decompositional Semantics Time) dataset. It is the largest temporal relations dataset to date, containing 32,302 events and 70,368 relations—significantly larger than previous benchmarks like TimeBank.

To build this dataset, the researchers used crowdsourcing to annotate the Universal Dependencies English Web Treebank. Annotators used "sliders" to indicate the relative start and end points of events on a normalized [0, 1] scale and selected from 11 duration classes ranging from "instant" to "forever."

Neural Architecture

The authors developed a joint model to predict both event duration and relative timing. The architecture utilizes ELMo embeddings followed by a "Tuner" layer and specialized attention mechanisms.

The neural architecture designed for joint prediction of event duration and relative temporal placement.
  • Event Model: Employs dot-product attention to generate vector representations of specific predicates.
  • Duration Model: Predicts likely durations using a Binomial distribution model. This enforces a "concave" probability curve, which the authors found outperformed a standard Softmax by assuming that if an event lasts "minutes," it is more likely to last "seconds" than "years."
  • Relation Model: Predicts four real-valued points—the start and end for two events—on the [0, 1] timeline.

Key Insights and Results

One of the paper's most compelling findings is that duration information is often encoded in non-temporal words. The model’s attention mechanism learned to identify implicit duration cues in nouns (e.g., "soldiers," "thoughts") and verbs.

Performance and Transfer Learning

Despite being trained on continuous scales, the model demonstrated impressive performance on traditional categorical tasks via transfer learning:

  • State-of-the-Art Results: It achieved an F1 score of 0.566 on TimeBank-Dense event-event relations, outperforming specialized systems like CAEVO (0.494) and CATENA (0.519).
  • High Correlation: The model showed a high Spearman correlation ($\rho \approx 0.78$) for predicting absolute start and end points on the UDS-T test set.
  • Linguistic Grounding: Analysis revealed that the model correctly attends to coordinators (e.g., "and," "or") and tense-bearing auxiliaries ("were," "are") to determine relative ordering.

Limitations and Future Directions

While the model excels at determining the order of events, it still faces challenges in predicting exact duration and placement (reflected in lower R1 scores). Furthermore, maintaining duration consistency across an entire document remains difficult. The authors suggest that future work should explore training models directly on more interpretable measures, such as "Priority" (who starts first) and "Containment" (does one event happen inside another), rather than raw coordinates.

Summary updated

Abstract

We present a novel semantic framework for modeling temporal relations and event durations that maps pairs of events to real-valued scales. We use this framework to construct the largest temporal relations dataset to date, covering the entirety of the Universal Dependencies English Web Treebank. We use this dataset to train models for jointly predicting fine-grained temporal relations and event durations. We report strong results on our data and show the efficacy of a transfer-learning approach for predicting categorical relations.
BibTeX
@inproceedings{vashishtha2019fine,
  author = {Siddharth Vashishtha and Benjamin Van Durme and Aaron Steven White},
  title = {{Fine-Grained Temporal Relation Extraction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  url = {https://openreview.net/forum?id=2TynTgmgOjC},
}
Figure