Unsupervised Storyline Extraction from News Articles

Deyu Zhou, Haiyang Xu, Xin-Yu Dai, Yulan He

In the era of 24-hour digital news, events do not occur in isolation; they evolve through complex, interconnected sequences. Extracting these "storylines"—such as the progression of an election or a multi-week international summit—is a critical task for making sense of high-volume news streams. This paper, presented at IJCAI 2016, introduces the Dynamic Storyline Extraction Model (DSEM), an unsupervised Bayesian framework designed to automatically identify and track the evolution of news stories over time.

The Challenge of News Evolution

Existing research in storyline extraction has historically struggled with three major hurdles:

  1. Error Propagation: Many systems extract events first and link them later. Errors in initial extraction compound, leading to incoherent storylines.
  2. Lack of Structure: Most models treat events as simple "bags of words," ignoring the structured reality that events involve specific actors (persons and organizations) and locations.
  3. Fixed Parameters: Many models require humans to pre-specify the number of storylines in a dataset, which is impractical for live, unpredictable news streams.

Methodology: The DSEM Framework

The authors propose DSEM, a generative model that jointly handles the "who, where, and what" of news. Unlike standard topic models, DSEM represents each storyline as a joint distribution over locations ($l$), organizations ($o$), persons ($p$), keywords ($w$), and latent topics ($z$).

The DSEM graphical model illustrating the joint distribution of entities, topics, and keywords, incorporating temporal dependencies and a switch mechanism.

Key Model Components

  • Structured Representation: By explicitly modeling entities, DSEM avoids confusing different contexts that might share similar vocabulary.
  • The "Switch" Mechanism: To filter noise, the model uses a per-token variable to determine if a word is a specific storyline keyword, a general background word, or a global topic word.
  • Temporal Evolution: The model captures story development by making the current epoch's probability distributions dependent on the previous $M$ epochs using an exponential decay function. This allows the model to "remember" context even during reporting lulls.
  • Non-parametric Discovery: Incorporating the Chinese Restaurant Process (CRP) allows the model to automatically spawn new storylines or assign documents to existing ones without human intervention.
  • Efficient Inference: To handle large-scale data, the authors implemented a per-token Metropolis-Hastings sampler, providing a 6x speedup over standard Gibbs sampling.

Experimental Evaluation

The researchers evaluated DSEM using the GDELT Event Database, utilizing three distinct datasets to test large-scale precision, ground-truth alignment (77 annotated storylines), and the ability to handle intermittent or short-term stories.

DSEM significantly outperformed baselines such as standard LDA, Dynamic LDA (DLDA), and the Recurrent Chinese Restaurant Process (RCRP). On the most complex dataset (Dataset III), which contained intermittent stories that disappear and reappear, DSEM maintained a high F-measure of 72.41%, while the previous state-of-the-art (SDM) dropped to 48.15%.

Qualitative Success: Structured Browsing

A key strength of DSEM is its ability to provide interpretable, structured summaries. The model successfully tracked the 2014 Indian Election, correctly identifying the shift from campaign rallies to exit polls and the eventual swearing-in ceremony of Narendra Modi.

Visualization of the 2014 Indian Election storyline extracted by DSEM, showing the evolution of entities and keywords over time.

Conclusion and Future Work

DSEM demonstrates that modeling the actors and locations of news is as important as modeling the vocabulary. While the model is robust, future work could explore multi-scale temporal dependencies to better capture both rapid and slow-moving stories, as well as improving entity resolution to mitigate errors from automated named entity recognizers. Despite these areas for growth, DSEM represents a significant step toward automated, high-fidelity news synthesis.

Summary updated

Abstract

Storyline extraction from news streams aims to extract events under a certain news topic and reveal how those events evolve over time. It requires algorithms capable of accurately extracting events from news articles published in different time periods and linking these extracted events into coherent stories. The two tasks are often solved separately, which might suffer from the problem of error propagation. Existing unified approaches often consider events as topics, ignoring their structured representations. In this paper, we propose a non-parametric generative model to extract structured representations and evolution patterns of storylines simultaneously. In the model, each storyline is modelled as a joint distribution over some locations, organizations, persons, keywords and a set of topics. We further combine this model with the Chinese restaurant process so that the number of storylines can be determined automatically without human intervention. Moreover, per-token Metropolis-Hastings sampler based on light latent Dirichlet allocation is employed to reduce sampling complexity. The proposed model has been evaluated on three news corpora and the experimental results show that it outperforms several baseline approaches.
BibTeX
@inproceedings{zhou2016unsupervised,
  author = {Deyu Zhou and Haiyang Xu and Xin-Yu Dai and Yulan He},
  title = {{Unsupervised Storyline Extraction from News Articles}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure