Learning Sentence-internal Temporal Relations

Mirella Lapata, Alex Lascarides

Introduction

Understanding the temporal ordering of events—determining whether one action occurs before, during, or after another—is a fundamental requirement for advanced natural language processing tasks such as multi-document summarization and question answering. While humans intuitively grasp temporal flow through context and implicit cues, machines struggle with these relations when they are not explicitly stated.

The primary obstacle in training models for temporal reasoning is the scarcity of annotated data. Traditional "gold standard" datasets, such as TimeBank, are meticulously annotated but remain small (under 70,000 words). This scale is often insufficient for robust machine learning. This research addresses this bottleneck by proposing a method to bypass manual annotation, instead utilizing the explicit temporal markers already present in massive, unannotated corpora as a training signal.

Methodology: Markers as Silver Labels

The authors frame temporal inference as a disambiguation task. By identifying eight specific temporal markers—after, before, while, when, as, once, until, and since—they extract sentences where these markers connect a main clause to a subordinate clause. These markers serve as "silver" labels, providing a proxy for the underlying temporal relationship.

Probabilistic Frameworks

The study evaluates two primary modeling approaches:

  1. Conjunctive Models: These capture direct dependencies between the two clauses, such as the specific interaction between the main verb and the subordinate verb.
  2. Disjunctive Models: These assume that the features of the two clauses are independent given the temporal marker. This approach is designed to mitigate data sparseness by reducing the complexity of the feature space.

Feature Engineering

To capture the "temporal signature" of a sentence, the authors developed a rich feature set including:

  • Linguistic Signatures: Tense, aspect, voice, and negation.
  • Lexical Semantics: Verb and noun identities, supplemented by semantic classes from WordNet "supersenses" and Levin verb classes.
  • Syntax: Tree complexity and the relative positioning of clauses.

An ensemble learning approach using the C5.0 decision-tree learner was employed to combine the outputs of these base models for final predictions.

Experimental Setup and Datasets

The research utilized two distinct datasets:

  • BLLIP Corpus: A 30-million-word collection of Wall Street Journal articles used for training. Approximately 84,000 instances were extracted where markers served as labels.
  • TimeBank: A high-quality, manually annotated corpus used for validation. The goal was to determine if a model trained on simple markers could predict expert-defined TimeML categories like BEFORE, INCLUDES, or SIMULTANEOUS.

Results and Analysis

The experimental results validate the hypothesis that explicit language can serve as an effective teacher for implicit relations.

Marker Prediction (Pseudo-disambiguation)

On the task of predicting a missing marker, the best ensemble model achieved 70.6% accuracy. This significantly outperformed the majority baseline (42.6%) and word-based baselines (48.2%). Notably, the model's agreement with the gold standard (47.5%) slightly exceeded the average inter-human agreement (45.0%), suggesting the model reached the "human ceiling" for this inherently ambiguous task.

Generalization to TimeBank

The most significant finding occurred during the transfer to the TimeBank corpus. When mapped to TimeML relations, the model trained on the large, "noisy" BLLIP data achieved 53.0% accuracy. In contrast, a model trained directly on the small, "clean" TimeBank data only achieved 42.7%. This demonstrates that data volume can effectively compensate for—and even surpass—data purity in temporal reasoning tasks.

Discussion

Several key insights emerged from the study:

  • The Power of Abstraction: Using verb classes rather than raw verb identities improved generalization, suggesting that temporal logic operates at the level of semantic categories.
  • Model Simplicity: Disjunctive models often outperformed conjunctive ones, highlighting how over-parameterization can hinder learning in high-dimensional linguistic spaces.
  • The Human Ceiling: Temporal relations are often subjective. The fact that the model approached human agreement levels indicates it successfully captured the majority of the learnable signal within the sentence structure.

Limitations and Future Directions

While successful, the current approach is limited to sentence-internal relations. It does not yet account for temporal flow across sentence boundaries or broader discourse contexts. Additionally, some markers (like since) are polysemous, introducing noise where the marker may indicate causality rather than time. Future work could integrate discourse-level features and coreference resolution to build a more holistic understanding of how events move through time across entire documents.

Conclusion

This research demonstrates a scalable path forward for temporal reasoning in NLP. By leveraging the "silver labels" provided by explicit markers in large-scale text, models can learn to infer implicit temporal relations with a level of proficiency that rivals human annotators and exceeds models trained on limited "gold" data.

Summary updated

Abstract

In this paper we propose a data intensive approach for inferring sentence-internal temporal relations. Temporal inference is relevant for practical NLP applications which either extract or synthesize temporal information (e.g., summarisation, question answering). Our method bypasses the need for manual coding by exploiting the presence of markers like ``after", which overtly signal a temporal relation. We first show that models trained on main and subordinate clauses connected with a temporal marker achieve good performance on a pseudo-disambiguation task simulating temporal inference (during testing the temporal marker is treated as unseen and the models must select the right marker from a set of possible candidates). Secondly, we assess whether the proposed approach holds promise for the semi-automatic creation of temporal annotations. Specifically, we use a model trained on noisy and approximate data (i.e., main and subordinate clauses) to predict intra-sentential relations present in TimeBank, a corpus annotated rich temporal information. Our experiments compare and contrast several probabilistic models differing in their feature space, linguistic assumptions and data requirements. We evaluate performance against gold standard corpora and also against human subjects.
BibTeX
@article{lapata2006learning,
  author = {Mirella Lapata and Alex Lascarides},
  title = {{Learning Sentence-internal Temporal Relations}},
  journal = {J. Artif. Intell. Res.},
  year = {2006},
  doi = {10.1613/jair.2015},
}
Figure