Introduction
In the landscape of social media analysis, research often focuses on the binary distinction between bots and humans. While identifying automation is critical for detecting large-scale manipulation, this binary fails to capture the nuance of social identity. A "human" account may belong to a private citizen, a professional journalist, or a government official—each possessing different levels of influence and social capital.
In the paper "Discover Your Social Identity from What You Tweet: a Content Based Approach," researchers Binxuan Huang and Kathleen M. Carley from Carnegie Mellon University propose a deep-learning framework to move beyond simple binary classification. By analyzing user biographies and tweet content through a hierarchical lens, they demonstrate that social roles can be identified automatically with high precision.
Problem Statement
Understanding the "sender's role" is foundational for disinformation research. Fake news spreads more effectively when amplified by celebrities or news agencies, yet current tools often struggle to distinguish between these high-influence categories. For instance, a news bot and a celebrity might both exhibit high posting frequencies, leading to false positives in traditional bot-detection systems.
The authors identify two primary gaps in current research: a lack of granularity in classification (often limited to verified vs. unverified) and a failure to effectively fuse profile metadata with longitudinal tweet history.
Methodology: Hierarchical Self-Attention
The authors propose a Hierarchical Self-Attention Neural Network designed to handle the inherent noise of Twitter data. The architecture integrates multiple layers to process text at different scales:
- Input Processing: To manage typos and slang, the model combines standard word embeddings with a Character-level Convolutional Neural Network (Char-CNN).
- Contextual Encoding: A Bi-LSTM layer captures the sequential context of words in both user bios and tweets.
- Hierarchical Attention Mechanism: This core component applies attention at three levels: word-level (identifying key terms like "legislation"), tweet-level (determining which specific tweets best define identity), and field-level (balancing the importance of the bio versus the tweet history).
Experimental Setup and Datasets
The study utilizes two distinct datasets to validate the model:
- Public Figure Dataset: A large-scale collection of approximately 500,000 accounts using Twitter’s "verified" status as a proxy for public figure status.
- Human-Labeled Identity Dataset: A refined dataset of ~11,000 users manually categorized into seven classes: News Media, News Reporter, Government Official, Celebrity, Company, Sport, and Regular People. This dataset achieved a high inter-rater agreement (0.96).
Results and Analysis
The "Full Model" significantly outperformed standard baselines, including SVMs and standard Bi-LSTMs. The hierarchical attention mechanism provided a ~2.2% increase in accuracy over flat attention models, proving that the structural relationship between words and tweets is vital for identity classification.
Key findings from the results include:
- The Dominance of Tweets: A user's tweet history is more discriminative of their identity than their profile description. While bios are often vague or aspirational, tweet content consistently reveals professional roles.
- Transfer Learning Efficacy: By pre-training on the massive "Public Figure" dataset and fine-tuning on the smaller human-labeled set, the model achieved high performance even with limited labeled data, boosting the Macro-F1 score from 86.09 to 88.63.
- Interpretability: Visualizing attention weights confirmed that the model prioritizes relevant keywords (e.g., "congressman" for government officials), providing a level of transparency to the classification process.
Conclusion and Future Work
While the model is robust, the authors note limitations regarding "silent" accounts with sparse content and the challenges of multi-faceted identities (e.g., a doctor who is also an entrepreneur). Future work aims to incorporate network features, such as follower graphs and profile imagery, to further refine the detection of social roles in the fight against disinformation.
Summary updated