#WhoAmI in 160 Characters? Classifying Social Identities Based on Twitter Profile Descriptions

Anna Priante, Djoerd Hiemstra, Tijs A. van den Broek, Aaqib Saeed, Michel L. Ehrenhard, Ariana NeedNLP+CSS @ EMNLP 2016 · DOI · OpenReview

Introduction

Social psychology has long established that identity is a primary motivator for collective action. However, bridging the gap between theoretical frameworks and large-scale data analysis remains a challenge. While Natural Language Processing (NLP) excels at predicting demographic traits like age or gender, it often overlooks the nuanced "social identity"—the self-concept derived from group memberships and social roles. This paper addresses this gap by developing a pipeline to automatically classify social identities within the 160-character constraints of Twitter bios.

Methodology

The researchers established a two-stage experimental pipeline grounded in identity and self-categorization theories. They defined five primary identity categories: Relational (e.g., family roles), Occupational (e.g., professional titles), Political, Ethnic/Religious, and Stigmatized (e.g., health conditions or self-deprecating labels).

A "gold standard" dataset was created by manually labeling 2,633 profiles from two populations: Movember participants (n=1,611) and a random sample of English-speaking Twitter users (n=1,022). Human annotation achieved high inter-rater reliability (Krippendorff’s alpha > 0.8). Classification was performed using four algorithms—Support Vector Machines (SVM), Bernoulli Naive Bayes (BNB), Logistic Regression (LR), and Random Forest (RF)—utilizing TF-IDF weighting and Chi-Square feature selection for the top 1,000 informative words.

Data Distribution and Identity Salience

Analysis of the manually labeled data revealed that users most frequently define themselves through "doing" and "belonging." Occupational identities were the most prevalent, appearing in 36% of profiles, followed by Relational identities at 28%. In contrast, political and religious identities were significantly rarer in the brief bio format, often accounting for less than 5% of the sample.

Distribution of social identity categories across the Movember and Random user datasets.

Results and the PES Merger

Initial experiments (Experiment 1) showed that while classifiers performed well for high-frequency categories like Occupational (F-score ~0.80) and Relational (F-score ~0.72), they struggled with the sparser "action-oriented" categories (Political, Ethnic/Religious, and Stigmatized), where F-scores frequently dipped below 0.50 for models like Bernoulli Naive Bayes.

To address this data sparsity and align with social psychological theory, the researchers implemented a merger (Experiment 2), combining these three categories into a single PES (Political-Ethnic-Stigmatized) category. This move was grounded in the concept that these identities all signal commitment to a collective group rather than an individual role. Following this merger, Logistic Regression (LR) emerged as the most robust classifier, achieving a more acceptable F-score of 0.623.

Performance metrics for the merged Political-Ethnic-Stigmatized (PES) category across different classifiers.

Discussion and Conclusion

The study demonstrates that social theory can effectively guide machine learning design. The difficulty in distinguishing between Political and Ethnic identities via NLP suggests these categories may be more consolidated in practice than traditional theory assumes.

Limitations include the 160-character limit, which may not capture a user's most salient identity, and potential sampling bias toward frequent tweeters. Future work will explore more sophisticated techniques, such as word embeddings (Word2Vec) and neural networks, and incorporate network features to provide a more holistic view of online social identity.

Summary updated

BibTeX
@inproceedings{priante2016whoami,
  author = {Anna Priante and Djoerd Hiemstra and Tijs A. van den Broek and Aaqib Saeed and Michel L. Ehrenhard and Ariana Need},
  title = {{\#WhoAmI in 160 Characters? Classifying Social Identities Based on Twitter Profile Descriptions}},
  booktitle = {NLP+CSS @ EMNLP},
  year = {2016},
  doi = {10.18653/v1/w16-5608},
}
Figure