Identifying and characterizing user communities on Twitter during crisis events

Aditi Gupta, Anupam Joshi, Ponnurangam KumaraguruDUBMMSM 2012 · DOI

TL;DR

This paper introduces a method to automatically identify and characterize user communities on Twitter during crisis events. By analyzing user links, content, and metadata, they cluster users and then pinpoint "top users" within each community who accurately represent the community's overall opinions and topics. This allows for more efficient monitoring of large-scale social media discussions during emergencies.

What Problem Does This Paper Address?

The sheer volume of information generated on social media platforms like Twitter during crisis events makes manual monitoring and analysis practically impossible. While these platforms are valuable sources of real-time information, understanding the diverse opinions and topics discussed within specific user groups is a significant challenge. This paper aims to address this by developing a framework to automatically group users into communities and identify key individuals within those communities who can serve as proxies for the group's collective sentiment and focus.

Methods

The authors propose a three-step methodology:

  1. Similarity Metric Definition: They introduce a novel similarity metric that combines three aspects of user interaction:
  2. Content Similarity: Based on common words, hashtags, and URLs in tweets related to the crisis event (after removing stop words).
  3. Link Similarity: Measures how often users retweet, mention, or reply to each other, or interact with common third-party tweets.
  4. Meta-data Similarity: Utilizes the "location" field in user profiles, validating it with Yahoo's PlaceFinder API to assess similarity at country, state, or city levels. These individual similarity scores are normalized and summed to create an overall user similarity matrix.
  5. Community Detection: Spectral clustering is applied to the user similarity matrix to group users into distinct communities. For their preliminary study, they intuitively set the number of clusters to three.
  6. Top User Identification: Degree centrality from social network analysis is used to identify the most influential users within each detected community.

Datasets

The study uses data collected from Twitter during three major crisis events in 2011:

  • England Riots: Over 1.1 million tweets were collected.
  • Hurricane Irene: Approximately 90,000 tweets were collected.
  • Earthquake in Virginia: Around 277,000 tweets were collected.

Data was gathered using Twitter's Streaming API. The paper does not introduce a new benchmark dataset but rather analyzes real-world crisis event data.

Key Insights and Ideas

  • Proxy Representation: The core idea is that a small set of "top users" within a community can effectively represent the collective opinions and topics of the entire community. This significantly reduces the amount of data that needs to be analyzed.
  • Multi-faceted Similarity: Combining content, link, and meta-data for user similarity provides a more robust understanding of user relationships than relying on a single factor.
  • Spammer vs. Informational Clusters: The analysis revealed that communities can effectively separate genuine information sharing from spam or promotional content, even when spammers leverage trending hashtags.
  • Trustworthiness Differentiation: Even within communities that discuss similar topics, analyzing the top users can help differentiate between more official/trustworthy sources (like news media) and general users.

Key Results

  • Community Quality: Spectral clustering yielded good quality communities, as indicated by modularity scores above 0.5 for all three events (Hurricane Irene = 0.67, England Riots = 0.51, Virginia Earthquake = 0.53), suggesting a strong division of users into distinct groups.
  • Top User Accuracy: The identified top central users (based on degree centrality) accurately represented the topics and opinions of their respective communities with an average accuracy of 81% (with a maximum of 96%). This was validated using the Mean Average Precision (MAP) metric.
  • Event-Specific vs. General Tweets: The high accuracy for event-specific tweets dropped significantly (to 32% on average) when tested on a random sample of general tweets, highlighting the unique characteristics of crisis-related discussions that enhance community coherence.
  • Qualitative Analysis: For Hurricane Irene, one cluster was identified as containing spammers, while others contained official news and alert profiles versus general users, demonstrating the method's ability to categorize community types.

Limitations and Future Work

  • Number of Clusters: The number of clusters was set intuitively to three for this preliminary study. Future work should involve applying standard techniques for automatic cluster number selection.
  • Similarity Metric Enhancement: The authors plan to strengthen their similarity metric by including more features and assigning weights to existing features.
  • Broader Event Analysis: The framework is generic and can be applied to any event. They aim to analyze more crisis events to further validate their approach.
  • Trustworthiness Metrics: While they showed a distinction between official and general users, further research into more sophisticated trustworthiness metrics for identified top users is suggested.

Summary updated

BibTeX
@inproceedings{gupta2012identifying,
  author = {Aditi Gupta and Anupam Joshi and Ponnurangam Kumaraguru},
  title = {{Identifying and characterizing user communities on Twitter during crisis events}},
  booktitle = {DUBMMSM},
  year = {2012},
  doi = {10.1145/2390131.2390142},
}
Figure