Temporal Reasoning on Implicit Events from Distant Supervision

Ben Zhou, Kyle Richardson, Qiang Ning, Tushar Khot, Ashish Sabharwal, Dan Roth

Abstract

Understanding narratives requires inferring implicit events that are not explicitly stated. This paper introduces TRACIE, a benchmark for evaluating temporal reasoning on these latent events. We also present SYMTIME, a neuro-symbolic model that combines neural distance and duration predictions with symbolic constraints. SYMTIME significantly outperforms standard language models, particularly in zero-shot settings and low-resource scenarios, by enforcing logical consistency in event end-time predictions.

Introduction

Temporal reasoning is essential for human-like story understanding. While most existing benchmarks focus on explicit events (e.g., "A happened, then B"), real-world comprehension involves "reading between the lines" to infer implicit states or actions. For example, if a driver fails to brake in time, we infer a preceding state of distraction.

Figure 1: Overview of the TRACIE task, illustrating the latent timeline of explicit and implicit events in a narrative.

Current language models struggle with these implicit events, largely because they lack a robust understanding of event durations and end-times. Natural language frequently omits end-times, creating a data gap that standard neural architectures fail to bridge.

The TRACIE Benchmark

The authors introduce TRACIE (Temporal Reasoning on Implicit Events), a dataset of 5.4k instances in a Textual Entailment (TE) format. TRACIE challenges models to verify temporal hypotheses about implicit events within short narrative contexts derived from ROCStories. This benchmark highlights the "End-Time Gap," where models are significantly worse at reasoning about when events end compared to when they start.

Methodology

The proposed SYMTIME model addresses temporal reasoning through two key innovations:

  1. Distant Supervision (PTNTIME): To overcome the scarcity of labeled temporal data, the authors use distant supervision. They extract millions of temporal patterns from Wikipedia and Project Gutenberg, allowing the model to learn relative distances (e.g., "weeks before") from noisy but abundant text.
  2. Neuro-Symbolic Reasoning: SYMTIME decomposes the temporal ordering problem into neural sub-tasks—predicting start-time distances and event durations—which are then combined using a symbolic rule.
Figure 5: The SYMTIME architecture, combining neural encoders for duration and distance with a symbolic logic layer.

By calculating whether Event A ends before Event B starts using the formula $\text{Distance}(\text{Start}_A, \text{Start}_B) + \text{Duration}(A)$, the model ensures that its predictions remain logically consistent with the principles of interval algebra.

Results and Discussion

SYMTIME achieved 80.6% accuracy on TRACIE, a substantial improvement over the T5-Large baseline (67.9%). Notably, in zero-shot evaluations, SYMTIME outperformed baselines by 11%, demonstrating that its symbolic constraints help it generalize beyond dataset-specific biases. Furthermore, pre-training on PTNTIME yielded a 9-point gain on the MATRES benchmark for explicit events in low-resource settings.

Despite these gains, limitations remain. The model's symbolic rules can struggle with zero-length gaps between events, and "reporting bias" in text can lead to skewed duration estimates. Future work aims to extend this reasoning to "multi-hop" timelines involving complex interactions between multiple events.

Conclusion

By explicitly modeling event durations and start-time distances, SYMTIME provides a more robust framework for temporal reasoning. The success of TRACIE and SYMTIME highlights the importance of symbolic constraints and distant supervision in capturing the latent temporal logic of human narratives.

Summary updated

BibTeX
@inproceedings{zhou2021temporal,
  author = {Ben Zhou and Kyle Richardson and Qiang Ning and Tushar Khot and Ashish Sabharwal and Dan Roth},
  title = {{Temporal Reasoning on Implicit Events from Distant Supervision}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.107},
}
Figure