Abstract
In social networking, the rapid detection of burst topics is vital for emergency response. However, social data is often noisy and sparse, evolving across time and space, making instant semantic capture difficult for traditional methods. This paper introduces FBTD (Fast Burst Topic Detect), a method that aligns data prediction with characteristic calculation to detect burst terms in their budding stages. By integrating local and global topic detection, FBTD maintains sub-second processing delays while ensuring high topic quality, outperforming state-of-the-art approaches.
Introduction
Social media platforms like Twitter and Weibo serve as real-time sensors for global events. Detecting "burst topics"—sudden surges in specific discussions—is critical for crisis management and market analysis. However, the inherent noise (spam, irrelevant posts) and sparsity of social data present significant challenges. Traditional models like Latent Dirichlet Allocation (LDA) are computationally heavy and often identify topics only after they have already peaked, losing the window for early intervention.
Methodology
The FBTD framework operates through a multi-stage pipeline designed for speed and early detection.
1. Burst Term Detection: The "Physics" of Word Frequency
FBTD treats term frequency as a dynamic physical system. To handle noise, it employs "evolving curve fitting" and secondary exponential smoothing to predict future trends. By calculating the velocity (rate of change) and acceleration (rate of the rate of change) of term frequencies, the system identifies "budding" terms before they reach high volume. This shift from monitoring volume to monitoring acceleration allows FBTD to catch the "spark" of a topic in its earliest stages.
2. Spatial-Temporal Integration and Topic Discovery
Once burst terms are identified, FBTD utilizes a grid-based approach to capture spatial and temporal context. The system partitions data into "geo-grids" and "time-grids," constructing directed weighted graphs where nodes represent terms and edges denote co-occurrence.
By extracting strongly connected components from these graphs, FBTD identifies local event clusters. These snapshots are then merged into global spatial-temporal topics using "virtual documents." This approach drastically reduces the computational load, allowing the algorithm to maintain a complexity of essentially $O(1)$ in practical scenarios.
Experimental Evaluation
The method was tested using a massive dataset provided by SINA Corp (Sina Weibo), containing over 500,000 data points related to real-world events like rainstorms, vaccine controversies, and public incidents in Kunming and Tianjin. All data included Unix timestamps and Geo-tags for spatial-temporal analysis.
Key Results
- Speed: FBTD maintains an internal delay of approximately 0.1 seconds, with total uptime under 1 second on standard hardware, significantly faster than baseline methods like TopicSketch.
- Timeliness: In head-to-head comparisons, the system identified burst topics one to two time units earlier than the BBTM baseline.
- Quality: Using Pointwise Mutual Information (PMI) as a metric for topic coherence, FBTD consistently outperformed LDA and BTM. Its results remained stable even as the time window size changed, whereas baseline methods saw performance drops when data became too sparse.
Conclusion
FBTD effectively hides the complexity of topic discovery by using data prediction to trigger analysis preemptively. While the model is highly effective, future work will address the sensitivity of prediction windows and explore deeper semantic integration, such as word embeddings, to better handle synonyms and sarcasm without sacrificing the $O(1)$ speed requirement.
Summary updated