TL;DR
When predicting whether a user will repost content on X (formerly Twitter), models relying solely on the post's text completely collapse when encountering new, unseen topics. By shifting the focus to stable user characteristics—such as profile metadata, social ties, and historical interaction patterns—prediction models remain highly robust under distribution shifts, revealing that viral sharing is largely content-agnostic.
What Problem Does This Paper Address?
Predicting whether a user will share a post is crucial for understanding how information spreads online, with major implications for marketing, political campaigns, and combating misinformation. Traditionally, machine learning models tackle this as an "in-distribution" problem: they are trained and tested on data covering the same set of topics.
However, social media is a firehose of rapidly evolving trends, breaking news, and novel hashtags. In the real world, a model trained on yesterday's topics must regularly predict user reactions to completely unseen topics today. This creates a challenging distribution shift. The authors set out to determine which signals remain reliable when predicting reposts for entirely new topics: the specific content of the post itself, or the stable characteristics of the users involved?
Methods
The authors frame reposting prediction as an individual-level binary classification task: given a post ($M$), a sender ($U_S$), and a recipient ($U_R$), will the recipient repost it?
To systematically isolate what drives these decisions, they categorized features into two main groups:
- Post-Related Features (M): 78 features extracted from the post text, including topics (via LDA), sentiment, emotional metrics, readability, and language style.
- User-Related Features (U): Split into user profiles (e.g., follower counts, verification status), historical actions (e.g., past repost rates, interaction frequencies), and historical message text.
They evaluated these feature sets across four distinct model architectures: a Decision Tree (XGBoost), a Multi-Layer Perceptron (MLP), BERT, and a Large Language Model (Qwen3-0.6B). To feed structured tabular user data into text-based LLMs like BERT and Qwen, they used a natural-language serialization strategy (e.g., converting raw metrics into sentences like "Follower count is 3774 and 915").
Datasets
The study utilized a dataset collected via the Twitter API containing the 14 most popular trending hashtags globally between July 27 and August 14, 2022.
- Core Data: 63,336 posts, which included 44,014 positive repost events.
- User Context: Data from 54,299 unique users, including their profile metadata, friend lists, and up to 50 historical posts per user (totaling over 2.6 million historical tweets).
- Negative Sampling: For every positive repost event, the authors selected the five most similar non-reposted posts containing the same hashtag within a 24-hour window to serve as rigorous negative controls.
Key Insights and Ideas
- Reposting is Content-Agnostic Under Shift: The most striking "aha moment" is that what is written in a post matters very little when generalizing to new topics. Social media users appear to have deeply ingrained sharing habits driven by who they are and who they follow, rather than the exact semantic nuances of a new topic.
- The Text-Feature Trap: Relying on post text creates a brittle model. Text-only models overfit to the specific vocabularies, entities, and contexts of the training topics, leading to a total performance collapse when a new topic emerges.
- Not All User Data is Equal: Interestingly, user historical message text (U-HM) also suffers under topic shifts. The truly transferable signals are structural and behavioral: network relationships (does the recipient follow the sender?) and historical action rates (how frequently does this user click repost in general?).
Key Results
The models were evaluated using the F1 score across several settings, with the most critical being the Out-of-Distribution (OOD) setup (training on 13 hashtags and testing on the 14th held-out hashtag):
- In-Distribution Success: When the training and testing topics overlapped, combining all features yielded strong results across all models (F1 scores between 0.78 and 0.88).
- OOD Collapse for Content Models: Under a topic shift, models relying only on post-related features (M) completely failed, with F1 scores plummeting to approximately 0.12—performing worse than random guessing (which sits at 0.167 for this dataset's class ratio).
- OOD Robustness for User Models: In stark contrast, models using only user-related features (U) maintained strong performance under topic shifts, achieving F1 scores above 0.70. Remarkably, the user-only models often performed as well as, or better than, models using all features combined.
Limitations and Future Work
While the paper strongly advocates for user-centric modeling, the authors note a few limitations:
- Text-Only Modality: The study only analyzed the message text of posts, ignoring images, videos, and external links, which are increasingly dominant on modern social platforms.
- Correlation vs. Causation: The results provide predictive evidence but do not map out the exact causal mechanisms behind user decisions.
- Algorithmic Confounders: The data reflects what users saw, meaning the platform's underlying recommendation algorithm heavily influences the observed behavior. Future work is needed to disentangle user psychology from algorithmic exposure.
Summary updated