The Significance of User Characteristics for Reposting Prediction on X: A Comparative Analysis Under Distribution Shift

Ziming Xu, Shi Zhou, Vasileios Lampos, Ingemar J. CoxarXiv 2025 · arXiv

TL;DR

When predicting whether a user will repost content on X (formerly Twitter), models relying solely on the post's text completely collapse when encountering new, unseen topics. By shifting the focus to stable user characteristics—such as profile metadata, social ties, and historical interaction patterns—prediction models remain highly robust under distribution shifts, revealing that viral sharing is largely content-agnostic.

What Problem Does This Paper Address?

Predicting whether a user will share a post is crucial for understanding how information spreads online, with major implications for marketing, political campaigns, and combating misinformation. Traditionally, machine learning models tackle this as an "in-distribution" problem: they are trained and tested on data covering the same set of topics.

However, social media is a firehose of rapidly evolving trends, breaking news, and novel hashtags. In the real world, a model trained on yesterday's topics must regularly predict user reactions to completely unseen topics today. This creates a challenging distribution shift. The authors set out to determine which signals remain reliable when predicting reposts for entirely new topics: the specific content of the post itself, or the stable characteristics of the users involved?

Methods

The authors frame reposting prediction as an individual-level binary classification task: given a post ($M$), a sender ($U_S$), and a recipient ($U_R$), will the recipient repost it?

To systematically isolate what drives these decisions, they categorized features into two main groups:

  • Post-Related Features (M): 78 features extracted from the post text, including topics (via LDA), sentiment, emotional metrics, readability, and language style.
  • User-Related Features (U): Split into user profiles (e.g., follower counts, verification status), historical actions (e.g., past repost rates, interaction frequencies), and historical message text.

They evaluated these feature sets across four distinct model architectures: a Decision Tree (XGBoost), a Multi-Layer Perceptron (MLP), BERT, and a Large Language Model (Qwen3-0.6B). To feed structured tabular user data into text-based LLMs like BERT and Qwen, they used a natural-language serialization strategy (e.g., converting raw metrics into sentences like "Follower count is 3774 and 915").

Datasets

The study utilized a dataset collected via the Twitter API containing the 14 most popular trending hashtags globally between July 27 and August 14, 2022.

  • Core Data: 63,336 posts, which included 44,014 positive repost events.
  • User Context: Data from 54,299 unique users, including their profile metadata, friend lists, and up to 50 historical posts per user (totaling over 2.6 million historical tweets).
  • Negative Sampling: For every positive repost event, the authors selected the five most similar non-reposted posts containing the same hashtag within a 24-hour window to serve as rigorous negative controls.
Figure 2: Example of a reposting event on X and the corresponding data sources from which different types of features are extracted.

Key Insights and Ideas

  • Reposting is Content-Agnostic Under Shift: The most striking "aha moment" is that what is written in a post matters very little when generalizing to new topics. Social media users appear to have deeply ingrained sharing habits driven by who they are and who they follow, rather than the exact semantic nuances of a new topic.
  • The Text-Feature Trap: Relying on post text creates a brittle model. Text-only models overfit to the specific vocabularies, entities, and contexts of the training topics, leading to a total performance collapse when a new topic emerges.
  • Not All User Data is Equal: Interestingly, user historical message text (U-HM) also suffers under topic shifts. The truly transferable signals are structural and behavioral: network relationships (does the recipient follow the sender?) and historical action rates (how frequently does this user click repost in general?).

Key Results

The models were evaluated using the F1 score across several settings, with the most critical being the Out-of-Distribution (OOD) setup (training on 13 hashtags and testing on the 14th held-out hashtag):

  • In-Distribution Success: When the training and testing topics overlapped, combining all features yielded strong results across all models (F1 scores between 0.78 and 0.88).
  • OOD Collapse for Content Models: Under a topic shift, models relying only on post-related features (M) completely failed, with F1 scores plummeting to approximately 0.12—performing worse than random guessing (which sits at 0.167 for this dataset's class ratio).
  • OOD Robustness for User Models: In stark contrast, models using only user-related features (U) maintained strong performance under topic shifts, achieving F1 scores above 0.70. Remarkably, the user-only models often performed as well as, or better than, models using all features combined.

Limitations and Future Work

While the paper strongly advocates for user-centric modeling, the authors note a few limitations:

  • Text-Only Modality: The study only analyzed the message text of posts, ignoring images, videos, and external links, which are increasingly dominant on modern social platforms.
  • Correlation vs. Causation: The results provide predictive evidence but do not map out the exact causal mechanisms behind user decisions.
  • Algorithmic Confounders: The data reflects what users saw, meaning the platform's underlying recommendation algorithm heavily influences the observed behavior. Future work is needed to disentangle user psychology from algorithmic exposure.

Summary updated

Abstract

Understanding information diffusion on X (formerly Twitter) requires accurate modelling of reposting behaviour. Most existing work predicts reposting under in-distribution settings, where training and test data cover the same topics. This paper addresses a more realistic and challenging scenario: out-of-distribution prediction, i.e., forecasting reposting behaviour for new, previously unseen topics. We formulate the task at the individual level - predicting whether a specific user will repost a given post - and systematically compare the predictive power of post-related features, user-related features, and their combination across four representative models: Decision Tree, Multi-Layer Perceptron, BERT, and Qwen. Our experiments show that while post-related features perform well in-distribution, their performance declines drastically for unseen topics, with F1 scores falling to approximately 0.12. In contrast, user-related features - including user profiles, social relations, and historical behaviour - deliver strong and transferable performance, raising the F1 score to over 0.70. These results demonstrate that reposting decisions are largely content-agnostic: they are driven more by stable user characteristics than by the specific content of a post. Our findings highlight the value of user modelling for building robust prediction systems and provide new insights into the mechanisms that enable information to spread across different topics.
BibTeX
@misc{xu2025significance,
  author = {Ziming Xu and Shi Zhou and Vasileios Lampos and Ingemar J. Cox},
  title = {{The Significance of User Characteristics for Reposting Prediction on X: A Comparative Analysis Under Distribution Shift}},
  year = {2025},
  eprint = {2505.15370},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.15370},
}
Figure