Understanding the Demographics of Twitter Users

Alan Mislove, Sune Lehmann, Yong‐Yeol Ahn, Jukka‐Pekka Onnela, Jan Rosenquist

TL;DR

This paper tackles the crucial question of whether Twitter users are representative of the general U.S. population. By analyzing a massive dataset of tweets and user profiles, the authors develop novel methods to infer demographics (geography, gender, race/ethnicity) and reveal that Twitter users are a significantly non-uniform sample, overrepresenting populous areas and exhibiting a male bias.

What Problem Does This Paper Address?

Online social networks like Twitter offer an unprecedented window into human communication and behavior at scale. However, a fundamental challenge in leveraging this data for broader societal insights is understanding who is actually on these platforms. If the user base is not representative of the general population, any analyses or predictions derived from Twitter data will inherently be biased. This paper addresses this gap by taking the first systematic steps to characterize the demographics of Twitter users and compare them to the U.S. population, aiming to quantify this sampling bias.

Methods

The authors developed several techniques to infer user demographics from the limited, self-reported data available on Twitter profiles:

  • Geographic Distribution: They used the self-reported "location" field in user profiles. To make this data mappable, they queried the Google Maps API for latitude and longitude coordinates. This process identified mappable locations for a significant portion of users. They then aggregated these locations to the U.S. county level to compare with U.S. Census data.
  • Gender Inference: Gender was inferred using the first word of a user's self-reported name. They compiled lists of the most popular male and female baby names from the U.S. Social Security Administration and matched user names against these lists.
  • Race/Ethnicity Inference: Similar to gender, race/ethnicity was inferred using the self-reported last name. They correlated user last names with race/ethnicity distributions provided by the U.S. Census for names with over 100 individuals in the 2000 Census.

Datasets

  • Twitter Data: A dataset of over 1.7 billion tweets from approximately 55 million users collected between March 2006 and August 2009.
  • U.S. Census Data: Used for comparison of geographic distribution (2000 Census) and race/ethnicity distributions (2000 Census).
  • U.S. Social Security Administration Data: Used to compile lists of popular male and female baby names (1900-2009) for gender inference.
  • U.S. National Atlas and U.S. Geological Survey Data: Used for mapping latitude/longitude to U.S. counties.

Key Insights and Ideas

  • Twitter is Not a Mirror: The most striking insight is that Twitter users are a highly non-uniform sample of the U.S. population, meaning analyses based on Twitter data will likely reflect these demographic biases.
  • Urban Bias is Strong: Populous counties are significantly overrepresented on Twitter compared to their proportion in the general population. This suggests that urban dwellers and those in densely populated areas are more likely to be Twitter users.
  • Gender Skew Exists, But is Shrinking: There's a notable male bias in the Twitter user base, particularly among early adopters. However, this bias appears to be decreasing over time, indicating a potential shift towards a more balanced gender representation.
  • Race/Ethnicity is Geographically Dependent: The distribution of race/ethnicity among Twitter users is not uniform across the country and shows distinct geographic patterns, with some groups being underrepresented in certain regions and overrepresented in others.

Key Results

  • Geographic Distribution: Twitter users are more likely to reside in populous counties. The median per-county Twitter representation rate is 0.324%, but this rate is significantly higher in populous counties, showing an order of magnitude difference between counties with 1,000 people and those with 1,000,000 people. The Midwest is notably underrepresented.
  • Gender: Gender could be inferred for 64.2% of U.S.-based users. Of these, 71.8% were identified as male. The male bias was significantly stronger among early adopters but decreased over time.
  • Race/Ethnicity: Race/ethnicity could be inferred for 71.8% of users. The paper highlights geographic trends, such as the undersampling of Hispanic users in the Southwest and African-American users in the South/Midwest, and the oversampling of Caucasian users in major cities.

Limitations and Future Work

  • Limitations:
    • Self-Reported Data: The accuracy of demographic inference is limited by the accuracy and completeness of self-reported user profile information (name, location).
  • Name Ambiguity: Inferences based on names can be imperfect due to name variations, misrepresentations, or names common to multiple demographics.
  • Location Granularity: Google Maps API can sometimes interpret locations at a coarser granularity than a U.S. county, requiring manual removal of broader location strings.
  • Per-User vs. Per-Tweet Data: Location data is per-user, not per-tweet, meaning a user's current location might not reflect the location of all their tweets if they've moved.
  • Race/Ethnicity Ambiguity: The last name-to-race/ethnicity mapping is not perfectly predictive, limiting direct comparison to Census distributions.
  • Future Work: The authors suggest that their analysis provides a foundation for future work. By understanding these sampling biases, researchers can immediately improve existing prediction and measurement methods by applying post-hoc corrections. Further nuanced analyses of Twitter population biases will enhance its utility as a sophisticated inference tool for understanding society.

Summary updated

Abstract

Every second, the thoughts and feelings of millions of people across the world are recorded in the form of 140-character tweets using Twitter. However, despite the enormous potential presented by this remarkable data source, we still do not have an understanding of the Twitter population itself: Who are the Twitter users? How representative of the overall population are they? In this paper, we take the first steps towards answering these questions by analyzing data on a set of Twitter users representing over 1% of the U.S. population. We develop techniques that allow us to compare the Twitter population to the U.S. population along three axes (geography, gender, and race/ethnicity), and find that the Twitter population is a highly non-uniform sample of the population.
BibTeX
@inproceedings{mislove2021understanding,
  author = {Alan Mislove and Sune Lehmann and Yong‐Yeol Ahn and Jukka‐Pekka Onnela and Jan Rosenquist},
  title = {{Understanding the Demographics of Twitter Users}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/icwsm.v5i1.14168},
}
Figure