Burst Topic Detection in Real Time Spatial-Temporal Data Stream

Chuangying Zhu, Junping Du, Qiang Zhang, Ziwen Zhu, Lei ShiIEEE Access 2019 · OpenReview

Abstract

In social networking, the rapid detection of burst topics is vital for emergency response. However, social data is often noisy and sparse, evolving across time and space, making instant semantic capture difficult for traditional methods. This paper introduces FBTD (Fast Burst Topic Detect), a method that aligns data prediction with characteristic calculation to detect burst terms in their budding stages. By integrating local and global topic detection, FBTD maintains sub-second processing delays while ensuring high topic quality, outperforming state-of-the-art approaches.

Introduction

Social media platforms like Twitter and Weibo serve as real-time sensors for global events. Detecting "burst topics"—sudden surges in specific discussions—is critical for crisis management and market analysis. However, the inherent noise (spam, irrelevant posts) and sparsity of social data present significant challenges. Traditional models like Latent Dirichlet Allocation (LDA) are computationally heavy and often identify topics only after they have already peaked, losing the window for early intervention.

Methodology

The FBTD framework operates through a multi-stage pipeline designed for speed and early detection.

1. Burst Term Detection: The "Physics" of Word Frequency

FBTD treats term frequency as a dynamic physical system. To handle noise, it employs "evolving curve fitting" and secondary exponential smoothing to predict future trends. By calculating the velocity (rate of change) and acceleration (rate of the rate of change) of term frequencies, the system identifies "budding" terms before they reach high volume. This shift from monitoring volume to monitoring acceleration allows FBTD to catch the "spark" of a topic in its earliest stages.

The FBTD pipeline for burst term detection, illustrating the transition from observed values to acceleration calculations.

2. Spatial-Temporal Integration and Topic Discovery

Once burst terms are identified, FBTD utilizes a grid-based approach to capture spatial and temporal context. The system partitions data into "geo-grids" and "time-grids," constructing directed weighted graphs where nodes represent terms and edges denote co-occurrence.

The spatial-temporal integration process, showing how geo-grids and time-grids are used to form virtual documents for topic detection.

By extracting strongly connected components from these graphs, FBTD identifies local event clusters. These snapshots are then merged into global spatial-temporal topics using "virtual documents." This approach drastically reduces the computational load, allowing the algorithm to maintain a complexity of essentially $O(1)$ in practical scenarios.

Experimental Evaluation

The method was tested using a massive dataset provided by SINA Corp (Sina Weibo), containing over 500,000 data points related to real-world events like rainstorms, vaccine controversies, and public incidents in Kunming and Tianjin. All data included Unix timestamps and Geo-tags for spatial-temporal analysis.

Key Results

  • Speed: FBTD maintains an internal delay of approximately 0.1 seconds, with total uptime under 1 second on standard hardware, significantly faster than baseline methods like TopicSketch.
  • Timeliness: In head-to-head comparisons, the system identified burst topics one to two time units earlier than the BBTM baseline.
  • Quality: Using Pointwise Mutual Information (PMI) as a metric for topic coherence, FBTD consistently outperformed LDA and BTM. Its results remained stable even as the time window size changed, whereas baseline methods saw performance drops when data became too sparse.

Conclusion

FBTD effectively hides the complexity of topic discovery by using data prediction to trigger analysis preemptively. While the model is highly effective, future work will address the sensitivity of prediction windows and explore deeper semantic integration, such as word embeddings, to better handle synonyms and sarcasm without sacrificing the $O(1)$ speed requirement.

Summary updated

Abstract

In the field of social network, fast detection of the burst topic plays a decisive role in emergency response and disposal. However, social data are noisy and sparse, which evolves with time going on and space changing make it difficult to catch the instant semantics with traditional methods. Instead of passively waiting for an emergency topic, we try to detect the latent burst topic in its budding stage. In this paper, we propose a fast burst topic detect method, namely, FBTD, which aligns data prediction with characteristic calculation to detect burst term from the real-time spatial-temporal data stream and integrates local topic detection with global topic detection to find the spatial-temporal burst topic. Our method controls the delay within a 0.1 s level while preserving the topic quality. The experiments show that preferable effects are procured, and our method outperforms the state-of-the-art approaches in terms of effectiveness.
BibTeX
@inproceedings{zhu2019burst,
  author = {Chuangying Zhu and Junping Du and Qiang Zhang and Ziwen Zhu and Lei Shi},
  title = {{Burst Topic Detection in Real Time Spatial-Temporal Data Stream}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=JBRFDfzdXd},
}
Figure