Abstract
The "SMP Challenge" aims to discover novel prediction tasks for numerous data on social multimedia and seek excellent research teams. Making predictions via social multimedia data (e.g. photos, videos or news) not only helps us to make better strategic decisions for the future, but also explores advanced predictive learning and analytic methods on various problems and scenarios, such as multimedia recommendation, advertising system, fashion analysis etc. In the SMP Challenge at ACM Multimedia 2019, we introduce a novel prediction task Temporal Popularity Prediction, which focuses on predicting future interaction or attractiveness (in terms of clicks, views or likes etc.) of new online posts in social media feeds before uploading. We also collected and released a large-scale SMPD benchmark with over 480K posts from 69K users. In this paper, we define the challenge problem, give an overview of the dataset, present statistics of rich information for data and annotation and design the accuracy and correlation evaluation metrics for temporal popularity prediction to the challenge.
Introduction
In the modern attention economy, predicting which content will gain traction is a critical challenge for advertisers, fashion analysts, and recommendation engine designers. However, social media popularity is notoriously fickle, influenced by a complex interplay of content, user profile, spatial context, and timing. Existing datasets often lack the scale or multi-modal richness required to capture these dynamics. Furthermore, most research has focused on static popularity rather than temporal evolution. The 2019 Social Media Prediction (SMP) Challenge addresses these gaps by formalizing the Temporal Popularity Prediction (TPP) task and providing the community with the Social Media Prediction Dataset (SMPD).
Methods and Evaluation
The TPP problem is defined as estimating the future impact—specifically view counts—of a post $v$ from user $u$ at a specific time $t$. To handle the high variance in raw view counts, the authors propose a log-normalization function to generate a "popularity score" $s$:
$$s = \log_2(\frac{r}{d} + 1)$$
where $r$ is the view count and $d$ is the number of days since the photo was posted. Performance is evaluated using two primary metrics: Spearman Ranking Correlation (SRC), to measure how well the model's predicted ranking of posts aligns with actual engagement, and Mean Absolute Error (MAE), to measure the numerical distance between predicted and actual popularity scores.
The SMPD Dataset
The centerpiece of the challenge is the SMPD benchmark, a massive multi-modal dataset comprising over 486,000 posts from 69,000+ Flickr users. Unlike previous datasets, SMPD provides a comprehensive view of each post, including visual features, textual metadata (titles and tags), and spatio-temporal data (GPS and timestamps).
The dataset is structured around a three-level hierarchical taxonomy. This taxonomy categorizes images into 11 broad categories (e.g., "Animal", "Urban") and 668 fine-grained sub-categories, ensuring a wide variety of real-world concepts.
Key Insights
- Temporal Context: Significant spikes in engagement were observed during October and December, highlighting the "Holiday Effect" and the importance of temporal context.
- Metadata Importance: Titles and tags (averaging 29 words per post) provide essential semantic context that complements visual data.
- Spatial Influence: Geographic data for 10% of items allows for the exploration of regional trends.
Results and Discussion
The dataset covers a continuous stream of 480 days, split chronologically into training (305k posts) and testing (181k posts) sets to ensure true predictive modeling. The average normalized popularity score was 6.41 in the training set and 5.12 in the test set.
While SMPD provides a robust benchmark, it is limited by its exclusive focus on Flickr, where user behavior may differ from platforms like TikTok or Instagram. Future work may expand to other interaction types (shares, comments) and address the "cold-start" problem for new users without historical data.
Summary updated