A Stratified Learning Approach for Predicting the Popularity of Twitter Idioms

Suman Maity, Abhishek Gupta, Pawan Goyal, Animesh Mukherjee

Abstract

Twitter Idioms are a distinct class of hashtags that facilitate conversational prompts and personal expressions. In this paper, we propose a two-stage stratified learning framework to predict their long-term popularity. Our approach first classifies hashtags into "Idiom" and "non-Idiom" categories with 86.9% accuracy. We then apply specialized regression models to the stratified samples, incorporating idiom-specific features. Experimental results demonstrate that this stratification significantly outperforms baseline methods, yielding improvements of 11.13% and 19.56% in correlation coefficients for popularity predictions at 7 and 11 months, respectively.

Introduction: The Unique Nature of Twitter Idioms

Predicting the trajectory of social media trends is a complex challenge. Traditional models often treat all hashtags as a homogeneous group, yet hashtags like #10ThingsAboutMe (an idiom) behave fundamentally differently from topical tags like #Election2013. Twitter Idioms are characterized by high emotionality, opinionated content, and distinct propagation patterns. By recognizing these nuances, we can develop more precise forecasting tools for social media dynamics.

The Stratified Learning Framework

The core of our methodology is a two-stage pipeline designed to handle the heterogeneity of hashtags.

The proposed two-stage stratified learning framework for hashtag popularity prediction.
  1. Stratification (Classification): We first employ a binary classifier (SVM or Logistic Regression) to separate incoming hashtags. Features used for this stage include hashtag content (length, POS diversity), tweet content (linguistic and psychological traits via LIWC), and user interaction patterns (retweet rates).
  2. Popularity Prediction (Regression): For hashtags identified as idioms, we utilize a Support Vector Regression (SVR) model. This stage introduces idiom-specific features such as "Hashtag Clarity" (word distribution distinctness) and "Early Coinage" (temporal momentum) to refine the prediction of future popularity.

Dataset and Methodology

Our study analyzed a massive dataset comprising a 1% random sample of the global Twitter stream from August 2011 to May 2014. We focused on English-language hashtags coined between January 2012 and May 2014 with a minimum frequency of 100. To establish ground truth, 9,000 hashtags were manually labeled, achieving over 90% inter-annotator agreement.

Key Insights and Experimental Results

Our analysis revealed that idiom-related tweets contain nearly twice the fraction of opinionated words compared to other hashtags, highlighting their emotional nature.

The classification stage achieved a robust 86.9% accuracy in distinguishing idioms. More importantly, the subsequent stratified regression showed substantial gains over the baseline.

Comparison of prediction performance (correlation ρ and error θ) across different time frames.

As shown in the table above, the inclusion of idiom-specific features led to an 11.13% improvement in the correlation coefficient (ρ) at the 7-month mark and a 19.56% improvement at the 11-month mark. Correspondingly, the Root Mean Square Error (RMSE) decreased by 8.4% and 6.73%, respectively.

Conclusion and Future Directions

This research demonstrates that "one size fits all" models are suboptimal for social media trends. While our model excels at long-term predictions for idioms, future work will focus on improving performance for non-idiom categories—which often depend on external news triggers—and increasing prediction granularity to a weekly resolution. Understanding these conversational idioms provides a vital window into community sentiment and the temporal scope of public opinion.

Summary updated

Abstract

Twitter Idioms are one of the important types of hashtags that spread in Twitter. In this paper, we propose a classifier that can stratify the Idioms from the other kind of hashtags with 86.93% accuracy and high precision and recall rate. We then learn regression models on the stratified samples (Idioms and non-Idioms) separately to predict the popularity of the Idioms. This stratification not only itself allows us to make more accurate predictions but also makes it possible to include Idiom-specific features to separately improve the accuracy for the Idioms. Experimental results show that such stratification during the training phase followed by inclusion of Idiom-specific features leads to an overall improvement of 11.13% and 19.56% in correlation coefficient over the baseline method after the 7th and the 11th month respectively.
BibTeX
@inproceedings{maity2021stratified,
  author = {Suman Maity and Abhishek Gupta and Pawan Goyal and Animesh Mukherjee},
  title = {{A Stratified Learning Approach for Predicting the Popularity of Twitter Idioms}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/icwsm.v9i1.14645},
}
Figure