TL;DR
This paper introduces CasMS, a novel framework for predicting the popularity of messages on social media. CasMS uniquely captures individual user retweeting preferences and integrates them with message diffusion models to achieve more accurate predictions across different stages of a cascade's lifecycle, from initial generation to long-term spread.
What Problem Does This Paper Address?
Predicting how widely a message will spread on social media (cascade popularity) is crucial for applications like viral marketing, content recommendation, and rumor detection. Existing deep learning methods often rely on observing the cascade's early stages, which limits their ability to predict popularity before a message is even posted. Furthermore, these methods tend to overlook the fact that different users have distinct preferences for what they retweet. This personalization is a key factor influencing cascade dynamics, yet it's largely ignored by current approaches. The paper aims to bridge this gap by developing a model that accounts for both personalized user behavior and the inherent diffusion process of messages, enabling predictions at various stages.
Methods
The proposed framework, CasMS, is built upon four key components:
- Modeling Personalized Retweeting Behavior: This module captures how individual users' preferences for certain types of messages influence their retweeting actions. It leverages message content features (extracted using BERT) and user social influence features (derived from network embeddings via Skip-gram) to compute a "retweeting probability" for each user-message pair.
- Learning the Cascade Snapshot Feature: This component uses a Graph Convolutional Network (GCN) to model the diffusion process within a cascade. It integrates user embeddings, their activation states, personalized retweeting probabilities, and other hidden features to learn representations of the cascade at specific points in time. A key innovation here is the explicit inclusion of personalized retweeting behavior as input to the GCN, and the theorem proving the monotonic non-decreasing nature of activation probabilities under certain conditions.
- Learning Multi-Resolution Temporal Feature: To capture the temporal dynamics of cascades, a Temporal Convolutional Network (TCN) is employed. TCNs are chosen over LSTMs for their ability to capture longer and more comprehensive temporal dependencies. The GCN outputs are fed into the TCN to learn multi-resolution temporal features.
- Joint Prediction Module: This final module uses Multi-Layer Perceptrons (MLPs) to take the learned temporal cascade features and predict the popularity increment. The framework is trained end-to-end, optimizing for Mean Squared Logarithmic Error (MSLE).
Datasets
The paper evaluates CasMS on four real-world datasets:
- Twitter: Contains tweets and retweets from March-April 2012.
- Weibo2016: Collected from Sina Weibo in China on July 1, 2016.
- APS: Represents citation relationships from the American Physical Society (APS) before 2017, where cascades are defined by citation behavior.
- Weibo2022: A newly collected dataset from Sina Weibo between December 2022 and June 2023.
For all datasets, 70% were used for training, 15% for validation, and 15% for testing.
Key Insights and Ideas
- Personalization is Key: The study strongly suggests that individual user preferences for retweeting specific content significantly impact cascade popularity prediction accuracy, a factor often overlooked by prior work.
- Unified Prediction Across Stages: CasMS demonstrates the ability to predict popularity not only from early cascade snapshots but also during the message generation stage (when no cascade data is available) and for both short-term and long-term horizons.
- TCNs for Temporal Dynamics: The choice of TCNs over LSTMs for temporal feature extraction proves beneficial, likely due to their superior ability to capture long-range dependencies in cascade evolution.
- GCN for Diffusion Modeling: The GCN effectively models the underlying diffusion process by integrating social graph structure with user and message features.
Key Results
- CasMS consistently outperforms 10 baseline methods across various datasets and prediction time periods, achieving an average improvement of over 10% in MSLE.
- In the message generation stage, CasMS significantly outperforms baseline methods like Feature-Linear and Feature-Deep, showcasing its ability to predict popularity without any initial cascade data.
- A case study on a Weibo2022 message with a burst cascade highlights CasMS's capability to accurately predict sudden surges in popularity, a feat missed by traditional smooth-increase prediction models.
- Ablation studies confirm the importance of each component: user feature extraction, personalized retweeting behavior modeling, and the TCN for temporal features all contribute significantly to the overall performance.
Limitations and Future Work
- Static Graphs: The current framework primarily accommodates static social graphs. The authors plan to generalize the model to handle dynamic, evolving graphs.
- Textual Content: The model currently focuses on textual communication. Future work will involve extending it to handle messages enriched with images and other multimedia content.
- Generalization: While the model shows strong performance, further research could explore its generalization capabilities across even more diverse social platforms and content types.
Summary updated