Inferring Human Traits from Facebook Statuses

Andrew Cutler, Brian Kulis

This paper, titled "Inferring Human Traits From Facebook Statuses," by Andrew Cutler and Brian Kulis, provides a comprehensive look at how our digital footprints—specifically the text we post—reveal deep-seated psychological and demographic traits.

TL;DR

Researchers demonstrate that simple Bag-of-Words (BoW) models can predict 20 different human traits, including IQ, religion, and political ideology, from Facebook status updates. The study sets new performance standards for several traits and introduces a novel "debiasing" technique to prevent models from using gendered language as a proxy for other characteristics.

What Problem Does This Paper Address?

While previous research has used Facebook "Likes" or basic demographics to predict user behavior, this paper focuses on the rich, unstructured text of status updates. The authors aim to bridge the gap between social science and machine learning by determining just how much "psychographic" information (personality, values, interests) can be extracted from everyday language.

Beyond mere prediction, the paper tackles the "black box" problem: what exactly are these models learning? By analyzing feature weights, the authors investigate whether models are picking up on meaningful behavioral signals or simply relying on demographic biases (like using "male" language to predict atheism).

Methods

The authors primarily utilize a Bag-of-Words (BoW) approach combined with tf-idf (term frequency-inverse document frequency) preprocessing.

  • Regression/Classification: They use $\ell_2$ regularized regression (Ridge Regression) for continuous traits and a "one-vs-one" majority vote system for multi-class categorical data.
  • Deep Learning Comparison: For gender prediction, they compared the BoW model against a 49-layer character-level Convolutional Neural Network (char-CNN). Surprisingly, the simpler BoW model outperformed the deep learning architecture.
  • Debiasing Technique: To mitigate "Disparate Mistreatment" (bias), they propose a method where a protected attribute (like gender) is explicitly encoded as a feature during training but set to "unknown" during testing. This forces the model to find predictive features that are independent of the protected attribute.

Datasets

The study uses the myPersonality dataset, a massive collection of data from over 7 million Facebook users who took psychological quizzes between 2008 and 2012.

  • Labels: The dataset includes scores for the "Big Five" personality traits, IQ (Raven’s Progressive Matrices), Satisfaction With Life (SWL), and the Sensational Interest Questionnaire (SIQ).
  • Scale: While millions took the quizzes, the authors focused on subsets of users who also shared their status updates (ranging from ~1,800 users for IQ to over 100,000 for gender).

Key Insights and Ideas

  • The Power of Simplicity: In the era of "deep learning everything," this paper finds that for social media text, simple linear models with high-dimensional BoW features are often more accurate and far more interpretable than complex neural networks.
  • IQ and Language: The model found that high-IQ users tend to use parsimonious, academic language (e.g., "physics," "final"), while lower-IQ scores were associated with more prosocial, relationship-oriented words (e.g., "family," "heart"). The authors note that Facebook statuses are currently a better predictor of IQ than a person's entire genetic profile.
  • The "Gamer" Proxy for Militarism: In analyzing traits relevant to political targeting (like those used by Cambridge Analytica), they found that "militarism" was highly predictable. However, the top features were simply "xbox" and "man," suggesting that complex psychographic models might sometimes just be expensive proxies for basic demographic and interest targeting.
  • Hidden Gender Bias: The authors discovered that an Atheist vs. Agnostic classifier was heavily reliant on gendered words (e.g., "fucking" for atheists, "beautiful" for agnostics). Their debiasing method successfully reduced this distortion without sacrificing overall accuracy.

Key Results

  • Gender Prediction: Achieved 92.8% accuracy, outperforming previous tri-gram models and the human baseline (84%).
  • IQ: The model accounted for 12.8% of the variance (EV) in IQ scores, a significant result for a purely text-based model.
  • Political Identity: On a binary Liberal vs. Conservative task, the model reached 76.4% accuracy. They also successfully expanded this to a 12-way classification across various ideologies.
  • Sensational Interests: Traits like "Militarism" and "Violent-Occult" interests were among the easiest to predict, even with relatively small training sets.

Limitations and Future Work

The authors acknowledge that their models only use status text; incorporating "Likes" and social network structure would likely increase accuracy. Additionally, the data is from 2008–2012, and social media language norms have evolved significantly since then.

Future work is suggested in transfer learning, particularly for traits with very few samples (like Satisfaction With Life), where a model trained on a data-rich trait (like personality) could be "fine-tuned" to improve predictions on rarer labels.

Summary updated

Abstract

This paper explores the use of language models to predict 20 human traits from users’ Facebook status updates. The data was collected by the myPersonality project, and includes user statuses along with their personality, gender, political identification, religion, race, satisfaction with life, IQ, self-disclosure, fair-mindedness, and belief in astrology. A single interpretable model meets state of the art results for well-studied tasks such as predicting gender and personality; and sets the standard on other traits such as IQ, sensational interests, political identity, and satisfaction with life. Additionally, highly weighted words are published for each trait. These lists are valuable for creating hypotheses about human behavior, as well as for understanding what information a model is extracting. Using performance and extracted features we analyze models built on social media. The real world problems we explore include gendered classification bias and Cambridge Analytica’s use of psychographic models.
BibTeX
@inproceedings{cutler2018inferring,
  author = {Andrew Cutler and Brian Kulis},
  title = {{Inferring Human Traits from Facebook Statuses}},
  booktitle = {SocInfo},
  year = {2018},
  url = {https://openreview.net/forum?id=Y8sgmBuKQCJ},
}

Loading related content...

Figure