Introduction
Social psychology has long established that identity is a primary motivator for collective action. However, bridging the gap between theoretical frameworks and large-scale data analysis remains a challenge. While Natural Language Processing (NLP) excels at predicting demographic traits like age or gender, it often overlooks the nuanced "social identity"—the self-concept derived from group memberships and social roles. This paper addresses this gap by developing a pipeline to automatically classify social identities within the 160-character constraints of Twitter bios.
Methodology
The researchers established a two-stage experimental pipeline grounded in identity and self-categorization theories. They defined five primary identity categories: Relational (e.g., family roles), Occupational (e.g., professional titles), Political, Ethnic/Religious, and Stigmatized (e.g., health conditions or self-deprecating labels).
A "gold standard" dataset was created by manually labeling 2,633 profiles from two populations: Movember participants (n=1,611) and a random sample of English-speaking Twitter users (n=1,022). Human annotation achieved high inter-rater reliability (Krippendorff’s alpha > 0.8). Classification was performed using four algorithms—Support Vector Machines (SVM), Bernoulli Naive Bayes (BNB), Logistic Regression (LR), and Random Forest (RF)—utilizing TF-IDF weighting and Chi-Square feature selection for the top 1,000 informative words.
Data Distribution and Identity Salience
Analysis of the manually labeled data revealed that users most frequently define themselves through "doing" and "belonging." Occupational identities were the most prevalent, appearing in 36% of profiles, followed by Relational identities at 28%. In contrast, political and religious identities were significantly rarer in the brief bio format, often accounting for less than 5% of the sample.
Results and the PES Merger
Initial experiments (Experiment 1) showed that while classifiers performed well for high-frequency categories like Occupational (F-score ~0.80) and Relational (F-score ~0.72), they struggled with the sparser "action-oriented" categories (Political, Ethnic/Religious, and Stigmatized), where F-scores frequently dipped below 0.50 for models like Bernoulli Naive Bayes.
To address this data sparsity and align with social psychological theory, the researchers implemented a merger (Experiment 2), combining these three categories into a single PES (Political-Ethnic-Stigmatized) category. This move was grounded in the concept that these identities all signal commitment to a collective group rather than an individual role. Following this merger, Logistic Regression (LR) emerged as the most robust classifier, achieving a more acceptable F-score of 0.623.
Discussion and Conclusion
The study demonstrates that social theory can effectively guide machine learning design. The difficulty in distinguishing between Political and Ethnic identities via NLP suggests these categories may be more consolidated in practice than traditional theory assumes.
Limitations include the 160-character limit, which may not capture a user's most salient identity, and potential sampling bias toward frequent tweeters. Future work will explore more sophisticated techniques, such as word embeddings (Word2Vec) and neural networks, and incorporate network features to provide a more holistic view of online social identity.
Summary updated