SMP Challenge: An Overview and Analysis of Social Media Prediction Challenge

Bo Wu, Peiye Liu, Wen-Huang Cheng, Bei Liu, Zhaoyang Zeng, Jia Wang, Qiushi Huang, Jiebo Luo

Abstract

Social Media Popularity Prediction (SMPP) is a crucial task that involves automatically predicting future popularity values of online posts, leveraging vast amounts of multimodal data available on social media platforms. Studying and investigating social media popularity becomes central to various online applications and requires novel methods of comprehensive analysis, multimodal comprehension, and accurate prediction. SMP Challenge is an annual research activity that has spurred academic exploration in this area. This paper summarizes the challenging task, data, and research progress. As a critical resource for evaluating and benchmarking predictive models, we have released a large-scale SMPD benchmark encompassing approximately half a million posts authored by around 70K users. The research progress analysis provides an overall analysis of the solutions and trends in recent years.

Introduction

In the modern digital landscape, predicting which content will gain traction is a high-stakes challenge for digital marketing, recommendation engines, and content creators. However, Social Media Popularity Prediction (SMPP) is notoriously difficult because popularity is influenced by a complex web of factors. These include multimodality (where an image’s impact is tied to its caption and tags), temporal dynamics (seasonal trends), and user context (historical reach). The SMP Challenge was established to provide a standardized, large-scale benchmark to address these complexities and foster innovation in predictive modeling.

The Social Media Prediction Dataset (SMPD)

The core contribution of this research activity is the Social Media Prediction Dataset (SMPD), a massive, unified collection sourced from Flickr. Flickr was selected for its rich metadata and high-quality imagery, providing a robust foundation for multimodal learning.

Components of a multimodal post in the SMPD dataset, including visual, textual, and temporal metadata.

The dataset consists of approximately 486,000 posts from 70,000 users. It includes:

  • Visual and Textual Content: Images paired with titles and over 250,000 custom tags.
  • Categorical Metadata: 756 category tags across 11 topics (e.g., nature, fashion), organized into a three-level hierarchy.
  • Spatio-temporal Data: 480 days of activity with precise upload timestamps and Point of Interest (POI) data for a subset of posts.

Methodology Evolution

The SMP Challenge has tracked a significant shift in predictive strategies over five years. The task is framed as a regression problem: predicting the log-normalized "view count" of a post.

Initially, standard approaches relied on Convolutional Neural Networks (CNNs) like ResNet-101 for image features, combined with ensemble learning techniques (e.g., LightGBM, XGBoost, or CatBoost) to fuse metadata. Recently, the field has transitioned toward Transformer-based architectures. Modern winning solutions utilize BERT or RoBERTa for text embeddings and Vision Transformers (ViT) for images. Advanced methods now employ contrastive learning to align titles and images—similar to the CLIP architecture—and curriculum learning to improve model generalization by training on progressively complex samples.

Key Insights and Data Analysis

Analysis of the SMPD benchmark has yielded several critical insights for the research community:

  1. Log-Normalization: Popularity typically follows a power-law distribution, where a small fraction of posts receive the majority of views. Applying a log function to normalize these values is essential for stable neural network training.
Distribution of post popularity scores within the dataset, demonstrating the power-law characteristics.
  1. The "Holiday Effect": Temporal analysis reveals that sharing activity peaks in October and December, corresponding with major holidays. Models that successfully incorporate these seasonal trends demonstrate superior performance.
  2. Textual vs. Visual Dominance: Baseline results indicate that text-based features (e.g., BERT) often outperform vision-only features (e.g., ResNext). This suggests that context and "searchability" via tags are primary drivers of initial views on platforms like Flickr.

Performance Benchmarks

The performance of predictive models has improved steadily as architectures have become more sophisticated. Early vision-only models achieved a Spearman Ranking Correlation (SRC) of 0.322, while text-only models reached 0.538. Multimodal fusion significantly boosted performance to 0.70. By the most recent iterations, state-of-the-art solutions reached an SRC of 0.77 with a Mean Absolute Error (MAE) of 1.24, highlighting the effectiveness of attention mechanisms in multimodal feature extraction.

Limitations and Future Directions

While the SMPD is a landmark dataset, it is limited by platform bias, as dynamics on "fast-feed" platforms like TikTok or X (formerly Twitter) may differ due to distinct algorithms and content lifespans. Additionally, location data remains sparse, with only 32,000 posts containing specific POI information. Future research should explore title-label contrastive learning and the generation of synthetic paired data to further bridge the gap between visual content and textual context.

Summary updated

Abstract

Social Media Popularity Prediction (SMPP) is a crucial task that involves automatically predicting future popularity values of online posts, leveraging vast amounts of multimodal data available on social media platforms. Studying and investigating social media popularity becomes central to various online applications and requires novel methods of comprehensive analysis, multimodal comprehension, and accurate prediction. SMP Challenge is an annual research activity that has spurred academic exploration in this area. This paper summarizes the challenging task, data, and research progress. As a critical resource for evaluating and benchmarking predictive models, we have released a large-scale SMPD benchmark encompassing approximately half a million posts authored by around 70K users. The research progress analysis provides an overall analysis of the solutions and trends in recent years. The SMP Challenge website (www.smp-challenge.com) provides the latest information and news.
BibTeX
@inproceedings{wu2023smp,
  author = {Bo Wu and Peiye Liu and Wen-Huang Cheng and Bei Liu and Zhaoyang Zeng and Jia Wang and Qiushi Huang and Jiebo Luo},
  title = {{SMP Challenge: An Overview and Analysis of Social Media Prediction Challenge}},
  booktitle = {ACM Multimedia},
  year = {2023},
  url = {https://openreview.net/forum?id=QvOYGZ2WN8_},
}
Figure