Abstract
Predicting the popularity dynamics of Twitter hashtags has a broad spectrum of applications. Existing works have mainly focused on modeling the popularity of individual tweets rather than the popularity of the underlying hashtags. Hence, they do not consider several realistic factors for hashtag popularity. In this paper, we propose Large Margin Point Process (LMPP), a probabilistic framework that integrates hashtag-tweet influence and hashtag-hashtag competitions, the two factors which play important roles in hashtag propagation. Furthermore, while considering the hashtag competitions, LMPP looks into the variations of popularity rankings of the competing hashtags across time. Extensive experiments on seven real datasets demonstrate that LMPP outperforms existing popularity prediction approaches by a significant margin. Going further, LMPP can accurately predict the relative rankings of competing hashtags, offering additional advantage over the state-of-the-art baselines.
Introduction
Predicting which hashtags will go viral is a critical task for viral marketing and trend detection. However, most existing models focus on individual tweet cascades rather than the hashtags themselves. This overlooks two vital factors:
- Reinforcement: A popular tweet can make a hashtag viral, and a trending hashtag can give a boost to a mediocre tweet.
- Competition: Twitter is a zero-sum game for attention. A breaking news hashtag (e.g., #NepalEarthquake) doesn't just grow in isolation; it often suppresses the visibility of other trending topics.
Existing temporal models often fail to capture these "microscopic" features, such as sudden rank shifts or the way different hashtags battle for dominance. The Large Margin Point Process (LMPP) aims to bridge this gap by creating a model that understands both why a hashtag grows and how it interacts with its rivals.
Methodology
The LMPP architecture is built on three pillars:
- Self-Exciting Dynamics: At its core, LMPP uses a Hawkes Process, a stochastic model where past events (tweets) increase the probability of future events.
- Hashtag-Tweet Reinforcement: Unlike standard models that treat all events equally, LMPP uses a generalized triggering kernel parameterized by a "tweet-popularity index." This acknowledges that the influence of a hashtag is more pronounced in popular tweets than in rare ones, assuming a power-law distribution for these indices.
- Inter-Hashtag Competition (The "Large Margin" Part): To model competition, parameter estimation is treated as a constrained optimization problem. Similar to a Support Vector Machine (SVM), the model introduces constraints that force it to maintain the correct popularity rankings between competing hashtags across time intervals. If the model predicts the wrong rank order, it is penalized. This "large margin" ensures robustness against sudden drifts in ranking.
Experimental Evaluation
The researchers evaluated LMPP on seven diverse real-world Twitter datasets, including entertainment (Oscars, MTV Awards), disasters (Nepal Earthquake), politics (Democratic Primaries), e-commerce (BBD), and sports (Copa America, T20 World Cup). These datasets vary significantly in scale and "rank diversity" (the frequency of hashtag ranking changes).
Performance Metrics
LMPP consistently outperformed five strong baselines (including RPP, SEISMIC, and SpikeM) across all datasets. As shown in the temporal analysis below, LMPP maintains superior performance over time in both error reduction and rank correlation.
Key findings include:
- Popularity Forecasting: LMPP achieved the lowest Mean Absolute Percentage Error (MAPE), showing improvements of up to 23.7% over the next best baseline (e.g., in the MTV-Awards dataset).
- Rank Prediction: Using Spearman’s Rank Correlation Coefficient (SRCC), LMPP proved significantly better at retrieving the actual popularity order of hashtags over time.
Jump Detection
In the difficult task of "catching" sudden popularity spikes (defined as a rank change of more than half the list), LMPP achieved an average precision and recall of approximately 70%, far exceeding the inconsistent performance of other models.
Conclusion
LMPP demonstrates that competition is a signal, not noise. By treating the rise of competing hashtags as a structural part of the environment and enforcing ranking margins, the model learns the relative attractiveness of hashtags more effectively than volume-only models. While the absolute number of tweets can be volatile, the relative ranking provides a stable signal for training. Future work may involve incorporating social network structures and developing online learning methods to adapt to emerging hashtags in real-time.
Summary updated