Analysis and Refinement of Temporal Relation Aggregation

Taylor Cassidy, Heng JiCOLING 2014 · OpenReview

Abstract

Building a complete timeline for entity relationships (e.g., employment or marriage) from scattered news articles is challenging because high-confidence errors can derail the aggregation process. This paper introduces three refinement methods—linguistic rules for titles, entity lifespans, and succession history—to clean temporal data before merging. These methods significantly improve the accuracy of temporal knowledge bases.

Introduction

In Information Extraction, the Temporal Slot-Filling (TSF) task aims to determine the start and end dates of "fluents"—properties of entities that change over time, such as a person's job title or a company's CEO. Because no single sentence typically provides a complete timeline, systems must aggregate fragments from multiple sources. However, current aggregation systems are brittle; a single mislabeled mention can "block" correct information from being recorded.

Problem Statement: The "WITHIN" Trap

The authors identified that "WITHIN" labels (indicating a relation holds at a specific time) are particularly dangerous. Because they are frequent and often assigned high confidence by classifiers, a single "WITHIN" error can extend a relation past its actual end date, poisoning the aggregate data. For example, if a person left office in 2012 but a vague text from 2015 is mislabeled as "WITHIN," the system may fail to record the correct 2012 end date.

Proposed Refinement Methods

To address these issues, the authors propose three pre-aggregation refinement techniques:

  1. Title Time of Predication Fix (TF): This method targets relational noun phrases like "former President." While standard NLP might assume the person is currently in office, TF uses hand-written rules to re-label these as AFTER_END*, providing a hard temporal bound.
  2. Entity Existence (EC): This uses external knowledge (e.g., Freebase, Wikipedia) as a sanity check. If a system claims a person held a role in 2010 but records show they died in 2009, the label is corrected to AFTER_END*.
  3. Relation Precedence (Pr): This leverages Wikipedia infoboxes (e.g., "preceded by"). If Person A succeeded Person B, information about Person B's tenure can constrain the start date for Person A.

Experimental Setup

The system was evaluated on the KBP 2013 TSF task using a corpus of 2.1 million documents. A significant contribution of this work is the manual analysis and correction of the original gold standard.

Analysis of invalidity sources in the original KBP 2013 gold standard.

The authors found 56 invalid entries in the original dataset, primarily due to wrong intermediate labels and conflicting information, as shown in the table above.

Results and Discussion

The refinements led to consistent performance gains across precision (P), recall (R), and F-score (F).

Performance comparison of the baseline system and incremental refinements.

While the numerical increases (from .314 to .336 F-score) may seem modest, the qualitative impact is substantial. These methods successfully "unblocked" correct information that was previously rejected due to high-confidence contradictions. By applying targeted linguistic rules and external constraints before aggregation, the system prevents errors from cascading.

Conclusion

This research demonstrates that linguistic nuance and world knowledge are vital for robust temporal aggregation. Future work will focus on better handling of nominal time of predication and automatically inducing chronological orders from raw text to reduce reliance on structured infoboxes.

Summary updated

BibTeX
@inproceedings{cassidy2014analysis,
  author = {Taylor Cassidy and Heng Ji},
  title = {{Analysis and Refinement of Temporal Relation Aggregation}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2014},
  url = {https://openreview.net/forum?id=ainPcU98G8F},
}
Figure