Real-time Scholarly Retweeting Prediction System

Zhunchen Luo, Xiao LiuCOLING 2018

Introduction

Twitter has evolved into a critical "early-warning system" for the scientific community, often serving as the first platform where researchers announce paper acceptances and breakthroughs. For academic institutions and individual researchers, the ability to predict which of these announcements will gain traction—measured by retweets—is vital for assessing real-time scientific impact. However, scholarly tweets are distinct from general social media content; they are semi-structured, often containing specific metadata such as author names, paper titles, and conference venues. This paper presents a real-time system designed to leverage these structural patterns and social network features to predict retweeting behavior with high accuracy.

System Architecture

The proposed system operates as a three-stage pipeline designed to process the Twitter firehose in real-time. The workflow begins with raw data ingestion and ends with a binary prediction of whether a tweet will be retweeted.

The three-stage system architecture including filtering, extraction, and prediction.

1. Scholarly Tweets Filter (STF)

The first stage utilizes a Support Vector Machine (SVM) to filter live streams (specifically those containing the phrase "paper accepted"). The goal is to separate genuine scholarly announcements (STs) from non-scholarly noise (NSTs). The filter analyzes features such as the user’s biography—cross-referencing academic affiliations via Wikipedia keywords—tweet length, and sentiment to ensure only relevant data enters the pipeline.

2. Tweet Scholar Block Extractor (TSBE)

The core innovation of this system is the Tweet Scholar Block Extractor. Recognizing that scholarly tweets follow specific conventions, the system treats the text as a sequence and employs a Conditional Random Field (CRF) with a BIO (Beginning, Inside, Outside) tagging schema. This allows the system to extract six distinct "Scholar Blocks" (TSBs): Author, Title, Venue, Time, Place, and Other.

3. Retweeting Predictor (RP)

The final stage uses another SVM to combine the extracted metadata blocks with social network features. These features include the author's social reach (follower and friend counts) and the historical popularity of the mentioned venue (e.g., the volume of tweets associated with a specific conference). By integrating content-specific metadata with social proof, the system makes its final prediction.

Experimental Results

The researchers evaluated the system using a custom dataset of 6,500 manually labeled tweets for filtering and 1,400 annotated scholarly tweets for extraction and prediction. The system was compared against several baselines, including random prediction and a deep learning model (SUA-ACNN).

Performance comparison showing the proposed system significantly outperforming deep learning and random baselines.

As shown in the results, the proposed system (TSBE+RP_Best) achieved an accuracy of 90.57%, significantly outperforming the deep learning baseline of 76.29%. When the system was provided with "Golden" (manually verified) metadata blocks, accuracy rose to 94.50%, indicating that while the automated CRF extractor is highly effective, minor error propagation remains a challenge for the end-to-end pipeline.

Key Insights and Conclusion

A significant finding of the study is the impact of social reach. Ablation studies revealed that the Sum Followers Count—the total reach of all authors mentioned in a tweet—is the most potent predictor of amplification. This suggests that in the academic sphere, existing social proof is a primary driver of information propagation.

By combining lightweight machine learning models (SVM and CRF) with structured metadata extraction, this system provides a computationally efficient solution for real-time scientific impact monitoring. Future work will focus on expanding the system's keyword triggers beyond "paper accepted" and further refining the sequence tagger to minimize error propagation between the extraction and prediction phases.

Summary updated

BibTeX
@inproceedings{luo2018real,
  author = {Zhunchen Luo and Xiao Liu},
  title = {{Real-time Scholarly Retweeting Prediction System}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure