Tracking Events Using Time-dependent Hierarchical Dirichlet Tree Model

Rumeng Li, Tao Wang, Xun Wang2015 · DOI

TL;DR

This paper introduces the time-dependent Hierarchical Dirichlet Tree (tHDT) model, a novel approach for generating news timelines that captures the hierarchical structure of topics within an event's evolution. By modeling these evolving topic hierarchies, tHDT improves sentence selection for timelines, leading to more coherent and informative summaries of long-term events.

What Problem Does This Paper Address?

Keeping up with long-term public events can be challenging due to the sheer volume of news and the complex, evolving nature of the topics involved. A single event often breaks down into numerous sub-topics that change over time, and these sub-topics are interconnected. Existing summarization techniques, particularly for timeline generation, struggle to effectively represent and leverage this hierarchical and dynamic topic structure. They often treat topics as flat or fail to capture the temporal dependencies and background information crucial for understanding an event's progression. This paper aims to bridge this gap by developing a model that can automatically learn and utilize these hierarchical topic structures for better timeline generation.

Methods

The core contribution is the time-dependent Hierarchical Dirichlet Tree (tHDT) model. This model extends the Dirichlet Tree distribution, which naturally models hierarchical topic structures, by incorporating time dependency and background information.

Here's a breakdown of the key methodological aspects:

  • Hierarchical Dirichlet Tree (tHDT): This model represents topics as a tree structure. Each sentence is assigned a path within this tree, reflecting its topical content at different levels of granularity.
  • Time Dependency: The model explicitly accounts for how topics evolve over time. This is achieved through a Markovian assumption, where the topic distribution at a given time is influenced by the distribution at the previous time step.
  • Background Information: The model can distinguish between background words and aspect-specific words within a topic. This is crucial for understanding the context and nuances of an event. For example, a word like "Lehman" might be a background word in a discussion about the "Financial Crisis" but an aspect word in the sub-topic "Collapse of Lehman Brothers."
  • Generative Process: The paper outlines a generative process for how sentences and topics are created within this hierarchical, time-dependent framework. This involves sampling paths through the tree and assigning words to topics.
  • Inference: A Gibbs sampler is used for model inference, allowing the model to learn the tree structure and topic distributions from data.
  • Sentence Selection: Once the topic structure is learned, sentences are selected based on a unified ranking framework that considers relevance, coherence, and coverage. The Jensen-Shannon divergence is used to measure sentence similarity.

Datasets

The authors created 8 datasets based on real long-term public events. These datasets were used for both training (parameter tuning) and testing. The paper notes that there are no existing standard evaluation datasets for timeline generation. The datasets were constructed by downloading news articles from various sources, and the details of the news sources and topics are provided in Tables 1 and 2.

Key Insights and Ideas

  • Hierarchy Matters: The most significant insight is that capturing the hierarchical structure of topics is crucial for effective timeline generation, especially for long-term events with evolving sub-topics. Flat topic models or those that don't consider this structure perform worse.
  • Time is Dynamic: Explicitly modeling time dependency and the evolution of topics over time leads to better timeline summaries. Simply ranking sentences without considering their temporal context is insufficient.
  • Background vs. Aspect: Differentiating between general background information and specific aspects of an event improves the model's ability to understand and represent the nuances of topic evolution.
  • Unified Ranking: Combining relevance, coherence, and coverage in a single sentence selection framework, guided by the learned topic hierarchy, is more effective than using individual criteria alone.

Key Results

The tHDT model, particularly with tree depths of L=2 and L=3, consistently outperformed various baseline methods, including traditional multi-document summarization (MDS) approaches (Centroid, GMDS), other timeline generation systems (Chieu, ETTS), and other topic models (t-HDP, HDT, D-LDA, LDA).

  • ROUGE Scores: The paper reports significant improvements in ROUGE scores (ROUGE-1, ROUGE-2, ROUGE-W) compared to baselines. For instance, tHDT(L=2) outperformed ETTS by 6.2%, 12.9%, and 11.7% in overall performance for ROUGE-1, ROUGE-2, and ROUGE-W, respectively.
  • Effectiveness of Components: Experiments showed that incorporating background information (tHDT vs. tHDT-noBG) and temporal information (tHDT vs. HDT) significantly boosted performance, validating the importance of these features.
  • Hierarchical Advantage: tHDT outperformed t-HDP and D-LDA, which treat topics as parallel, highlighting the benefit of modeling hierarchical topic structures.

Limitations and Future Work

  • Tree Depth: While L=2 and L=3 showed good performance, the optimal tree depth might vary depending on the event. The paper suggests that a 2 or 3 layer tree approximates the hierarchical structure of topics well.
  • Parameter Tuning: The paper mentions a gradient search strategy for tuning parameters like weights for relevance, coherence, and coverage (w1, w2, w3), decay factor (λ), and tree depth (L). However, the sensitivity and optimal values of these parameters for different types of events could be a subject for further exploration.
  • Automatic Tree Structure Learning: While the model learns topic distributions, the paper notes that the standard Dirichlet Tree model requires a predefined structure. Although their tHDT model can learn the structure across the corpus, further work could focus on more sophisticated automatic learning of the tree structure itself.
  • Query Expansion: The paper uses pseudo-relevance feedback for query expansion. Future work could explore more advanced query expansion techniques.

Summary updated

Abstract

Timeline Generation, through generating news timelines from the massive data of news corpus, aims at providing readers with summaries about the evolvement of an event. It is a new challenge of summarization that combines salience ranking with novelty detection. For a long-term public event, the main topic usually includes many different sub-topics at varying epochs, which also has its own evolving patterns. Existing approaches fail to utilize such hierarchical topic structure involved in the news corpus for timeline generation. In this paper, we develop a novel time-dependent Hierarchical Dirichlet Tree Model (tHDT) for timeline generation. Our model can aptly detect different levels of topic information in corpus and the structure is further used for sentence selection. Based on the topic distribution mined from tHDT, sentences are selected through an overall consideration of relevance, coherence and coverage. We develop experimental systems to compare different rival algorithms on 8 long-term events of public concern. The performance comparison demonstrates the effectiveness of our proposed model in terms of ROUGE metrics.
BibTeX
@misc{li2015tracking,
  author = {Rumeng Li and Tao Wang and Xun Wang},
  title = {{Tracking Events Using Time-dependent Hierarchical Dirichlet Tree Model}},
  year = {2015},
  doi = {10.1137/1.9781611974010.62},
}
Figure