Online Hot Topic Detection Based on Segmented Timeline and Aging Theory

Ruiguo Yu, Xiaodong Xie, Yongxing Li, Mankun Zhao, Xuyuan Dong, Muwen HeInternational Journal of Hybrid Information Technology 2016 · DOI

Abstract

With a great deal of digitized textual information now available on the internet, it is almost impossible for people to assimilate all the information timely. Therefore, the technologies of topic detection and tracking are used for constructing news topics from news stories in order to bring convenience to people. However, traditional topic detection methods are not always so effective in detecting emerging hot news topics in a short period of time, and most topic detection methods use single-pass clustering algorithm which is with low accuracy and very sensitive to the input sequence of news documents. In order to improve clustering accuracy, we utilize a temporal distance factor to segment timeline into equal parts and propose a novel two-times single-pass clustering algorithm to deal with news stories in each part of timeline separately. Moreover, the aging theory is combined with our approach to build life-span model of topics from which we can obtain variation trend of hotness value of topics. The results of experiments show that our approach is effective and the life circle model of topics established by our method can conform to reality well.

Introduction

The modern internet landscape is characterized by an "information explosion," where the sheer volume of news stories makes manual tracking impossible. Topic Detection and Tracking (TDT) systems aim to solve this by automatically grouping related stories. However, traditional TDT systems often rely on single-pass clustering, which, while efficient, is highly sensitive to the order in which documents arrive. If related stories are separated by a large temporal gap or a high volume of unrelated noise, the system may fail to link them, leading to fragmented topics.

To address these challenges, this research proposes a refined approach that combines multi-stage clustering with biological metaphors to identify "burst" topics—those that emerge suddenly and dominate the news cycle.

Methodology

Two-Times Single-Pass Clustering

The core innovation is a two-tier clustering strategy designed to reduce sensitivity to document sequencing:

  1. Timeline Segmentation: The news stream is divided into equal "time distances" (e.g., 24-hour blocks). This aligns the algorithm with the natural daily rhythm of news consumption.
  2. Stage 1 (Subtopic Clustering): Within each time block, news stories are clustered into subtopics using a standard single-pass algorithm.
  3. Stage 2 (Global Topic Clustering): These subtopics are then clustered again across different time blocks to form global topics. This ensures that related stories are captured even if they appear in different segments of the timeline.

Title-Pivot Incremental TF-IDF

Recognizing that headlines often contain the most salient information, the researchers implemented a weighting factor (λ) that gives terms appearing in the title a 30% boost in importance during the vectorization process. This helps the system focus on the primary subject matter rather than auxiliary details in the body text.

Aging Theory and Hotness Modeling

To model the "hotness" of a topic, the system treats topics like living organisms.

  • Nutrition: New, similar stories provide "nutrition" to a topic, increasing its "energy" (hotness value).
  • Decay: Over time, if no new stories appear, the topic's energy naturally decays based on a constant factor (β).

This allows the system to visualize the life-span of an event from its "birth" (emergence) to its "death" (fading from public interest).

Experimental Setup and Datasets

The researchers collected a dataset of 83,000 news stories from five major Chinese news portals (including Sina and 163.com) over an eight-day period in March 2014. For evaluation, 8,300 stories were randomly selected, and 8 specific topics were manually labeled as ground truth to validate the system's accuracy.

Table 1: The 8 manually labeled ground truth topics used for system validation, including their story counts.

As shown in the table above, the topics ranged from ongoing international crises like the situation in Ukraine to discrete events like the Oscar Awards. An additional 10,000 documents were used to establish initial document frequencies for the TF-IDF model.

Key Results and Insights

The proposed method demonstrated a significant advantage over traditional single-pass baselines:

  • Recall Improvement: The average Recall jumped from 0.732 to 0.917. By clustering subtopics in a second pass, the system effectively eliminated the fragmentation that often plagues standard TDT algorithms.
  • F-Measure: The overall F-measure (balancing precision and recall) improved from 0.770 to 0.884.
  • Life-Cycle Visualization: By applying energy functions, the researchers successfully plotted "life-cycle curves." For instance, the "Kunming terrorist attack" showed a rapid energy spike followed by a steady decay, while the "Ukraine crisis" maintained high energy without decaying, reflecting its status as an ongoing conflict.

Conclusion

By segmenting the timeline and applying a two-stage clustering process, this approach significantly improves the robustness of hot topic detection. The integration of Aging Theory provides a realistic model for tracking the rise and fall of news events, offering a powerful tool for navigating the modern information landscape. Future work will focus on automated thresholding and using the "rapid-growth" phase of the life-cycle curves to predict which emerging subtopics will become major breaking news events.

Summary updated

Abstract

With a great deal of digitized textual information now available on the internet, it is almost impossible for people to assimilate all the information timely. Therefore, the technologies of topic detection and tracking are used for constructing news topics from news stories in order to bring convenience to people. However, traditional topic detection methods are not always so effective in detecting emerging hot news topics in a short period of time, and most topic detection methods use single-pass clustering algorithm which is with low accuracy and very sensitive to the input sequence of news documents. In order to improve clustering accuracy, we utilize a temporal distance factor to segment timeline into equal parts and propose a novel two-times single-pass clustering algorithm to deal with news stories in each part of timeline separately. Moreover, the aging theory is combined with our approach to build life-span model of topics from which we can obtain variation trend of hotness value of topics. The results of experiments show that our approach is effective and the life circle model of topics established by our method can conform to reality well.
BibTeX
@article{yu2016online,
  author = {Ruiguo Yu and Xiaodong Xie and Yongxing Li and Mankun Zhao and Xuyuan Dong and Muwen He},
  title = {{Online Hot Topic Detection Based on Segmented Timeline and Aging Theory}},
  journal = {International Journal of Hybrid Information Technology},
  year = {2016},
  doi = {10.14257/ijhit.2016.9.2.22},
}
Figure