TL;DR
This paper introduces a novel method for inferring Twitter user demographics by leveraging website traffic data as a form of "weak supervision." By correlating website visitor demographics with the Twitter followers of those websites, the authors build a regression model that predicts demographic attributes with surprising accuracy, even outperforming fully supervised methods in some cases.
What Problem Does This Paper Address?
Understanding the demographics of social media users is crucial for various applications, from targeted marketing to public health campaigns. However, obtaining accurate demographic information for large numbers of users is challenging. Traditional methods often rely on costly and error-prone manual annotation of user profiles or text, and many demographic variables (like income or education) are not explicitly stated by users. This paper tackles the problem of inferring demographic attributes of Twitter users without requiring individual user labels, by instead using aggregate, indirectly obtained demographic data.
Methods
The core innovation of this paper lies in its use of distantly supervised learning and website traffic data. Instead of labeling individual Twitter users, the authors leverage publicly available demographic data for millions of websites from Quantcast.com. They then construct a "neighbor vector" for each website, representing the Twitter followers of that website's account. This neighbor vector captures the network structure around the website's followers.
A multi-task elastic net regression model is then trained to predict the demographic profile of a website's visitors (obtained from Quantcast) using the website's neighbor vector as input. Elastic net is chosen for its ability to handle high-dimensional data and perform feature selection, while the multi-task variant ensures that related demographic variables (like different education levels) share similar feature selections.
For validation, the authors also explore a classification approach, repurposing the trained regression model to predict individual user demographics.
Datasets
- Quantcast.com Data: Demographic data for 1,532 websites, covering six variables: Gender, Age, Income, Education, Children, and Ethnicity. This data serves as the "weak supervision" source.
- Twitter API Data:
- Follower data for 1,066 Twitter accounts associated with the sampled websites.
- "Friends" (accounts followed by followers) of these users, creating a two-hop network.
- A manually annotated validation set of 770 Twitter users (for ethnicity and gender) collected via the Twitter Streaming API.
Key Insights and Ideas
- Website traffic data as a proxy for user demographics: The paper demonstrates that the demographic makeup of a website's visitors is strongly correlated with the demographic makeup of the Twitter users who follow that website's official account. This is a powerful insight for obtaining large-scale, indirectly labeled data.
- Network structure reveals demographics: The "neighbor vector" – representing who the followers of a website follow – is a surprisingly rich source of demographic information. This suggests that social connections are not random and can reveal underlying user attributes.
- Regression model outperforms supervised classification for gender: The authors found that their regression model, trained solely on aggregate data, achieved better performance for gender prediction than a logistic regression classifier trained on a smaller, hand-labeled dataset. This highlights the power of their weakly supervised approach.
- Scalability with limited data: The study shows that even a small number of followed accounts (as few as 10 per user) can provide a significant portion of the predictive accuracy, suggesting that their method can be scaled even with Twitter's API rate limits.
Key Results
- Aggregate Prediction: The regression model achieved an average held-out correlation of .77 across six demographic variables when predicting website visitor demographics from follower network data. Correlations ranged from .55 (35-44 age bracket) to .89 (Male and African American).
- Classification Performance: For gender classification, the regression-based approach outperformed a standard logistic regression classifier trained on labeled data. For ethnicity, the regression approach was competitive, outperforming classification until a significant amount of labeled data was used.
- Data Efficiency: The F1 score using only 10 friends per user was within 5% of the score obtained using 200 friends, indicating high data efficiency.
Limitations and Future Work
- Bias in Twitter Data: The authors acknowledge potential biases, such as Twitter users skewing younger than the general web traffic panel. They also note that their validation set for ethnicity might be biased towards users with identifiable ethnicity.
- Generalizability: The paper suggests testing the generalizability of their approach to new user groups, such as predicting demographics by city or county and comparing with Census data.
- Semi-Supervised Learning: They propose investigating ways to combine labeled and unlabeled data using semi-supervised learning techniques.
- Privacy Implications: The authors note that revealing friend identity information, even for a small number of friends, can reveal significant demographic information, raising privacy concerns for users.
Summary updated