Harvesting Parallel News Streams to Generate Paraphrases of Event Relations

Congle Zhang, Daniel S. WeldEMNLP 2013 · DOI

Introduction

Identifying semantically equivalent phrases, or paraphrases, is a fundamental challenge in Natural Language Processing (NLP). Traditional approaches typically rely on the distributional hypothesis, which posits that words appearing in similar contexts share similar meanings. While effective for many tasks, this approach often fails to distinguish between synonyms and antonyms (e.g., "rise" vs. "fall") or related but distinct events (e.g., "attack" vs. "defeat"), as they frequently occupy identical linguistic environments. Furthermore, these methods often require massive static corpora, making them ill-suited for capturing paraphrases of emerging or rare events.

To address these limitations, researchers at the University of Washington developed NEWSSPIKE. This system leverages "temporal burstiness" within parallel news streams—multiple outlets reporting on the same real-world events simultaneously—to identify high-precision paraphrases of event relations.

Methodology

The NEWSSPIKE algorithm moves beyond simple text similarity by utilizing time as a primary signal for event identity. The pipeline involves extracting (subject, relation, object) tuples using OpenIE and then applying a series of temporal heuristics encoded within a probabilistic graphical model.

Overview of the NEWSSPIKE pipeline, from news streams to paraphrase clusters.

The core methodology relies on three "Temporal Correspondence Heuristics":

  1. Temporal Functionality: Articles published at the same time mentioning the same entities with consistent verb tenses are likely describing the same event.
  2. Temporal Burstiness: The model identifies sudden spikes in the frequency of entity pairs. These spikes distinguish unique, specific news events from routine mentions (e.g., "Obama" and "White House"), providing cleaner candidates for paraphrasing.
  3. One Event-Mention Per Discourse: Following the "one sense per discourse" principle, the model assumes a single article will not use multiple distinct phrases to describe the exact same fact, serving as a useful inference constraint.

The system uses Integer Linear Programming (ILP) for joint inference, allowing these heuristics to reinforce each other across clusters of relations.

Experimental Setup

The system was evaluated using the NewsSpike Corpus, a dataset of 0.5 million timestamped news articles collected over 50 days. From this, 2.6 million tuples were extracted using the ReVerb OpenIE system. For benchmarking, the authors compared NEWSSPIKE against distributional methods like "Resolver" (trained on 1.8 million NYT articles) and deep learning metrics from Socher et al. (2011).

Results and Discussion

NEWSSPIKE demonstrated superior performance in both precision and the ability to discover lexically diverse paraphrases.

Comparison of NEWSSPIKE against baseline and state-of-the-art models.

Key findings include:

  • High Precision: NEWSSPIKE achieved a precision of 0.92, outperforming the Pairwise model (0.90) and the Socher et al. baseline (0.81).
  • Lexical Diversity: In "diverse" clusters—where paraphrases have different head verbs—NEWSSPIKE maintained a high precision of 0.87, whereas the distributional baseline (Resolver) dropped to 0.65.
  • Scalability: The system generated over 24,000 relation clusters, significantly more than distributional methods could produce at comparable precision levels.

While highly effective, the model can occasionally be misled by temporally coincident but non-synonymous phrases (e.g., "narrowly misses" vs. "flies close to"). Additionally, the focus on high precision naturally limits the overall recall of the system.

Conclusion

NEWSSPIKE demonstrates that temporal signals from parallel news streams are a powerful alternative to traditional distributional similarity for paraphrase extraction. By focusing on "bursty" events, the system achieves state-of-the-art precision and handles rare events more effectively than static corpus-based methods. Future work may involve hybridizing temporal models with distributional metrics to balance high precision with broader recall.

Summary updated

BibTeX
@inproceedings{zhang2013harvesting,
  author = {Congle Zhang and Daniel S. Weld},
  title = {{Harvesting Parallel News Streams to Generate Paraphrases of Event Relations}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2013},
  doi = {10.18653/v1/d13-1183},
}
Figure