TL;DR
This paper introduces the Compound Topic Model (CTM), a method for detecting "topic drift"—the way conversations evolve, emerge, or fade over time in document streams like blogs. By training a single topic model on combined "past" and "present" data, the authors show that simple statistical measures like Kullback-Leibler (KL) divergence can effectively flag significant shifts in public discourse, such as the rise of gas price concerns or the launch of the Android platform.
What Problem Does This Paper Address?
In the world of social media and news, topics aren't static; they are a moving target. While standard tools like Latent Dirichlet Allocation (LDA) are excellent at summarizing what a static collection of documents is about, they struggle to capture evolution.
The authors identify a specific need for "online" detection: knowing exactly when a new conversation starts or an old one dies out within a scrolling window of time. This is vital for marketing intelligence—for instance, spotting a sudden surge in complaints about a product or the emergence of a new competitor—without having to manually read thousands of posts.
Methods
The researchers propose a functional approach called the Compound Topic Model (CTM).
- The Model: Instead of training two separate models and trying to align them, they take two datasets ($D$ for the past and $D'$ for the present) and train a single LDA model on the union of both. This ensures that the "topics" are consistent across both time periods.
- Indicators of Drift: To quantify how much the conversation has changed, they tested three metrics:
- Relative Perplexity: How "surprised" a model trained on the past is by the words appearing in the present.
- Topic KL Divergence (TKLD): Measuring the difference between the distribution of topics in the past vs. the present.
- Unigram KL Divergence (UKLD): A simpler measure comparing the raw word frequencies between the two periods.
- Normalization: To make these scores useful, they normalized the KL divergence against its maximum possible value, creating an "absolute" measure of drift that isn't just relative to background noise.
Datasets
The authors evaluated their approach using both controlled and real-world data:
- Synthetic Data: 30,000 documents generated from 10 artificial topics over 300 days, used to test if the model could detect both gradual shifts and the sudden appearance of a brand-new topic.
- Toyota Weblogs (2008): A collection of blog posts mentioning "Toyota" from January to June 2008.
- iPhone Weblogs (2007–2008): Posts mentioning "iPhone" and "platform" from April 2007 to March 2008.
Key Insights and Ideas
- Unigrams as an Early Warning System: One of the most practical findings is that simple unigram-based indicators (UKLD) are often just as effective at detecting drift as complex topic models. This suggests a two-stage pipeline: use cheap unigram checks to find "interesting" time windows, and only then run the computationally expensive topic models to understand what changed.
- Unified Generative Space: By using a CTM, the authors avoid the "correspondence problem" (trying to figure out if Topic A in January is the same as Topic B in February). The CTM treats a change in an existing topic and the birth of a new topic as the same mathematical phenomenon.
- Correlation with Real-World Events: The model doesn't just find statistical noise; it finds history. The "drift" detected in the Toyota data perfectly tracked the record-breaking rise in US gasoline prices in mid-2008, showing a shift toward "hybrid" and "fuel" conversations.
Key Results
- Detection Accuracy: In synthetic tests, all three indicators (Perplexity, TKLD, and UKLD) successfully identified the exact window where topic drift was introduced.
- Toyota Case Study: The model identified a major drift peaking in June 2008. Analysis showed the most volatile topic was related to "gas, hybrid, fuel, prius," which nearly doubled in frequency as gas prices hit all-time highs.
- iPhone Case Study: The indicators peaked in November 2007. The CTM successfully extracted topics related to the Apple SDK announcement (October) and the subsequent rise of the "Android" and "Open Handset Alliance" (November), capturing the shift from hardware hype to platform competition.
Limitations and Future Work
- Parameter Sensitivity: The authors noted that smoothing parameters in the Gibbs sampling process could introduce a "lag" in detecting new topics, though they didn't explore this deeply.
- Fixed Windows: The study used a fixed 4-month window ($K=4$). The authors acknowledge that different window lengths would likely detect different "granularities" of drift (e.g., a 1-week window for viral memes vs. a 6-month window for economic shifts).
- Demographics: A suggested future direction is using CTMs to compare topic distributions across different demographics (age, gender, location) rather than just time.
Summary updated