Classifying and ranking microblogging hashtags with news categories

Shuangyong Song, Yao Meng

TL;DR

This paper tackles the challenge of organizing the free-form nature of microblogging hashtags by classifying them into news categories and then ranking their popularity within those domains. This approach helps users discover trending topics more effectively by providing a structured and context-aware view of hashtag discussions.

What Problem Does This Paper Address?

Microblogging platforms like Twitter (and in this paper, Sina Weibo) rely heavily on hashtags for topic organization and discussion aggregation. However, the "free style" creation of hashtags means they lack inherent domain information, making it difficult for users to navigate and find relevant content across different subject areas. Existing methods for hashtag analysis often overlook this crucial domain classification aspect, leading to a disorganized user experience. This paper aims to bridge this gap by introducing a method to classify hashtags into predefined news categories and then rank their popularity within these specific domains.

Methods

The proposed approach consists of three main modules:

  1. Training a Domain Classification Model: The authors leverage existing news websites (specifically SOHU news) that already categorize articles. They extract features from news articles using both a bag-of-words approach and a topic modeling approach. Two classification models are trained on these features, and their average performance is used as the final indicator. Support Vector Machines (SVM) are used for training in their preliminary experiments.
  2. Domain Classification of Hashtags: To classify a hashtag, the paper first needs to obtain its semantic text. Instead of directly using all tweets containing a hashtag (which can be noisy), they propose a clustering method with a self-adaptive threshold. This method clusters tweets based on their average Euclidean distance and selects the largest cluster as the representative text for the hashtag. Bag-of-words and topic-level features are then extracted from this semantic text, and the trained domain classification model is used to assign a domain distribution to the hashtag.
  3. Domain-Sensitive Ranking of Hashtags: To rank hashtags within their assigned domains, the paper considers three factors:
  4. The probability of the hashtag belonging to a specific domain (obtained from the classification model).
  5. The number of microblogs containing the hashtag.
  6. The recency of these microblogs, using a temporal decay function to represent fading user interest. A formula is provided to calculate a "domain hot value" based on these factors.

Datasets

  • News Articles: News articles from the SOHU news website were used for training the domain classification model. For each domain, 10,000 articles were collected.
  • Microblogging Data: A dataset of 2,000 randomly chosen hashtags from Sina Weibo was used for preliminary experiments. This dataset was collected using the public timeline API.
  • Human Annotation: Three graduate students were recruited to annotate the domain classification and domain-sensitive ranking of the hashtags, providing scores from 1 to 5 for hotness.

Key Insights and Ideas

  • Bridging News and Microblogging: The core idea of using structured news categories to bring order to the unstructured world of microblogging hashtags is a significant conceptual contribution.
  • Self-Adaptive Clustering for Semantic Text: The proposed method for extracting representative text for a hashtag by clustering tweets and using the largest cluster is a practical way to mitigate noise from irrelevant spam or off-topic posts.
  • Domain-Sensitive Popularity: Recognizing that a hashtag's popularity is not absolute but context-dependent (i.e., within a specific domain) is crucial for providing more meaningful insights.

Key Results

  • The classification precision of their model achieved 0.91.
  • For correctly classified hashtags, a NDCG@10 value of 0.898 was obtained, demonstrating the model's usefulness in describing hashtags.
  • Preliminary experiments on Sina Weibo showed that domains like 'Entertainment', 'Social', and 'Sports' attracted the most user interest and generated the most topics.
  • Human annotators showed substantial agreement on domain-sensitive hot value scores, with an average kappa score of 0.74.

Limitations and Future Work

  • Model Performance Comparison: The authors acknowledge that a more detailed comparison of different model performances (beyond SVM) will be considered in the future.
  • Keyword-Based Search: Future work includes designing a model for keyword-based hashtag search.
  • User Influence and Interest: The authors plan to develop a hashtag ranking mechanism that additionally considers user interest and user influence.
  • Daily Ranking: Implementing daily domain-sensitive hashtag ranking is also a planned future experiment.

Summary updated

Abstract

In microblogging, hashtags are used to be topical markers, and they are adopted by users that contribute similar content or express a related idea. However, hashtags are created in a free style and there is no domain category information about them, which make users hard to get access to organized hashtag presentation. In this paper, we propose an approach that classifies hashtags with news categories, and then carry out a domain-sensitive popularity ranking to get hot hashtags in each domain. The proposed approach first trains a domain classification model with news content and news category information, then detects microblogs related to a hashtag to be its representative text, based on which we can classify this hashtag with a domain. Finally, we calculate the domain-sensitive popularity of each hashtag with multiple factors, to get most hotly discussed hashtags in each domain. Preliminary experimental results on a dataset from Sina Weibo, one of the largest Chinese microblogging websites, show usefulness of the proposed approach on describing hashtags.
BibTeX
@inproceedings{song2015classifying,
  author = {Shuangyong Song and Yao Meng},
  title = {{Classifying and ranking microblogging hashtags with news categories}},
  booktitle = {RCIS},
  year = {2015},
  url = {https://openreview.net/forum?id=A1CfF__2EU},
}
Figure