TL;DR
This paper presents a domain-independent, machine-learning approach to automatically determine the temporal order of events in news articles. By leveraging linguistic features and a novel reference time calculation, the system achieves significant accuracy in both anchoring events to specific times and establishing their relative order, outperforming previous methods.
What Problem Does This Paper Address?
Understanding the temporal relationships between events is crucial for many natural language processing (NLP) tasks, such as summarizing multiple news articles or answering complex questions about past events. However, news articles often don't follow a strict chronological order; they prioritize "news value," meaning the most recent or impactful events might appear first. This makes it challenging for machines to accurately infer when events happened and in what sequence. Existing methods struggle with this inherent complexity, necessitating a more robust approach to temporal ordering.
Methods
The authors propose a domain-independent, machine-learning based approach that involves several key steps:
- Linguistic Feature Extraction: The system utilizes a suite of linguistic features extracted from each clause in a news article. These include:
- Clause Type: Identifying different clause structures (regular, relative, complement).
- Temporal Expressions (TIMEX2): Recognizing explicit time mentions like "June 1, 2001" or "Monday."
- Temporal Modifiers (TMOD): Detecting phrases like "after the altercation."
- Verb Tense and Aspect: Analyzing the grammatical tense (past, present, future) and aspect (progressive, perfect) of verbs.
- Reporting Verbs: Identifying verbs that indicate someone is speaking or reporting.
- Quoted Text: Handling events described within quotation marks.
- Reference Time (tval) Computation: A crucial innovation is the computation of a "reference time" (tval) for each clause. This tval is either an explicit temporal expression found in the clause or, if none exists, an implicit time inferred from the context. The algorithm for computing tval starts with the document creation date and iteratively updates it based on explicit times, clause types, reporting verbs, and quoted text.
- Anchor-Explicit Feature: This feature captures the temporal relationship between an event's time and its computed tval (e.g., at, before, after).
- Machine Learning for Anchoring and Ordering: The extracted linguistic features, including the computed tval and anchor-explicit information, are used to train a statistical classifier (C5.0 Rules). This classifier learns rules for:
- Event Anchoring: Determining the temporal relationship between an event and its reference time.
- tval Moves: Tracking how the reference time changes across successive clauses (keeping the same tval, reverting to a previous one, or shifting to a new one).
- Partial Ordering of Events: Based on the learned anchoring rules, the system generates "anchor tuples" for each document. These tuples, along with the ordered tvals, are then used to create a partial ordering of events within the document, represented as links between event pairs.
Datasets
The paper describes using a dataset of 2069 clauses extracted at random from the North American News Corpus for training and evaluation. A human annotator corrected the tval and anchored the event's time with respect to the tval for this sample. The authors also mention using TDT2-documents for inter-annotator reliability testing of their temporal expression tagger, TempEx.
Key Insights and Ideas
- The Narrative Convention is Often Broken in News: The pilot experiment revealed that news articles only adhere to chronological ordering (narrative convention) about 47% of the time, highlighting the need for sophisticated temporal inference.
- Reference Time (tval) is a Powerful Abstraction: The computed reference time for each clause proves to be a valuable feature for anchoring events, even when explicit temporal expressions are absent.
- Machine Learning Outperforms Majority Class: The statistical classifier trained on linguistic features significantly outperforms a simple majority-class prediction for both anchoring and tval move prediction.
- Linguistic Features are Key: A diverse set of linguistic features, including tense, aspect, and temporal adverbials, are essential for accurate temporal inference.
Key Results
- Event Anchoring Accuracy: The overall approach achieves 84.6% accuracy in temporally anchoring events.
- Partial Ordering Accuracy: The system achieves 75.4% F-measure in partially ordering events.
- tval Computation Accuracy: The initial algorithm for computing tval achieved 59% accuracy, with machine learning improving upon this.
- Anchor-Explicit Accuracy: The anchor-explicit feature was correct 83.8% of the time.
- Comparison to Prior Work: The results compare favorably to previous literature, with higher accuracy on anchoring and a more comprehensive approach to temporal ordering.
Limitations and Future Work
- Skewed Distributions: The authors acknowledge that news events are often dominated by "AT" (at the reference time) relationships, which can lead to skewed distributions in training data. Future work will examine learning from such distributions.
- Role of Aspectual Features: The paper suggests that further investigation into the role of aspectual features could improve performance.
- Unsupervised Learning: The incorporation of unsupervised learning methods is proposed as a future direction.
- Naïve tval Algorithm: While the naïve tval algorithm is only 59% correct, the authors note that improvements are possible, but they used the corrected tval for testing the machine-learned rules.
Summary updated