Sense-Level Semantic Clustering of Hashtags in Social Media

Ali Javed, Byung Suk Lee

Introduction

Hashtags are essential for indexing the massive, unstructured stream of social media content. However, clustering them into meaningful semantic groups is challenging. Traditional methods rely on tweet text, which is often sparse and noisy. Metadata-based approaches using dictionaries like WordNet offer a solution but typically operate at the word level. This paper introduces a "sense-level" approach that disambiguates multiple meanings (senses) of hashtags, achieving a 26% improvement in clustering accuracy over word-level methods.

The Problem: Sparsity, Noise, and Polysemy

Clustering hashtags based on tweet context faces two primary hurdles:

  1. Sparsity: Many hashtags lack sufficient textual context for robust analysis.
  2. Noise: Tweet language is often informal and linguistically poor.

While dictionary-based metadata can mitigate these issues, word-level clustering introduces the "false triangle" problem. Because many words are polysemous—having multiple meanings—unrelated hashtags can be incorrectly grouped if they share a multi-sense word. For example, the hashtag #date could refer to a fruit or a calendar day.

The False Triangle Problem: Word-level clustering (right) incorrectly assumes strong similarity between H2 and H3 because they both relate to H1, whereas sense-level analysis (left) reveals they connect to different meanings of H1.

Methodology

The proposed sense-level semantic clustering algorithm operates in three stages:

1. Semantic Grounding

Hashtags are mapped to specific "synsets" (sets of synonyms) in WordNet. For hashtags not present in WordNet, the algorithm queries Wikipedia categories to extract relevant nouns for matching. This step ensures that every hashtag is anchored to formal semantic definitions.

Example of polysemy in WordNet: The word 'desert' has multiple distinct senses (synsets) ranging from a noun for arid land to verbs meaning to abandon or defect.

2. Similarity Matrix Construction

Instead of a single similarity score, the algorithm calculates the similarity between every possible sense of two hashtags using the Wu-Palmer measure. It records the specific sense pair that yields the maximum similarity, creating a more granular and accurate similarity matrix.

3. Hierarchical Clustering

Using this sense-level matrix, the algorithm performs agglomerative (bottom-up) clustering. This allows a single hashtag to potentially appear in multiple clusters if it possesses distinct, relevant meanings (e.g., #apple appearing in both "Fruit" and "Technology" clusters).

Evaluation and Results

The authors evaluated the approach using the Symplur Dataset (1,010 hashtags) and a manually annotated Gold Standard Benchmark of 230 hashtags across 15 clusters.

The sense-level approach achieved a weighted average F-score of 0.43, a significant improvement over the 0.34 achieved by the word-level baseline. This represents a 26% relative gain in accuracy. In practical tests, the algorithm successfully disambiguated hashtags like #date, correctly grouping the fruit sense with #fruit and the temporal sense with #august, whereas word-level baselines merged them into a single incoherent cluster.

Conclusion and Future Work

By shifting from word-level to sense-level analysis, this research demonstrates that external formal knowledge (WordNet and Wikipedia) provides a more reliable semantic anchor for short-form social media data than tweet context alone.

Future work aims to incorporate informal metadata sources like Urban Dictionary to better capture internet slang and to expand multilingual support through automated translation services.

Summary updated

BibTeX
@inproceedings{javed2016sense,
  author = {Ali Javed and Byung Suk Lee},
  title = {{Sense-Level Semantic Clustering of Hashtags in Social Media}},
  booktitle = {SIMBig (Revised Selected Papers)},
  year = {2016},
  doi = {10.1007/978-3-319-55209-5\_1},
}
Figure