You Are What You Read: Inferring Personality From Consumed Textual Content

Adam Sutton, Almog Simchon, Matthew Edwards, Stephan LewandowskyACL 2023

TL;DR

This paper explores inferring personality from consumed text (what people read) rather than produced text (what people write). They found that traditional "message-level attention" models, successful for produced text, perform poorly on consumed text. A novel "average personality per article" approach significantly improves prediction accuracy, suggesting that the aggregate personality of an article's readers is a strong signal.

What Problem Does This Paper Address?

The paper tackles the challenge of inferring personality from the text people consume, a domain largely unexplored compared to inferring personality from produced text (like social media posts or emails). While previous work has shown that authored text can reveal personality traits (e.g., using linguistic analysis), it's unclear if the text we choose to read carries similar predictive power. This has implications for understanding and potentially mitigating psychological micro-targeting, especially in political advertising, where tailoring messages to personality can increase effectiveness. The authors aim to reverse-engineer this process to develop countermeasures.

Methods

The core of the paper lies in comparing two distinct approaches for personality inference from consumed text:

  • Message-Level Attention: This is a state-of-the-art method for produced text, where models learn to weigh the importance of individual messages (e.g., posts) from a user to predict their personality. The authors adapt this by encoding each consumed article into a vector and then applying a sequence model with attention.
  • Average Personality Per Article: This novel approach assigns a single personality label to each article. This label is the average personality score of all participants known to have consumed that article. The assumption is that an article implicitly targets a group with a discernible aggregate personality profile. This average label is then used to train a model to predict personality from the article's content.

Both methods utilize Sentence-BERT for text encoding and feed-forward neural networks for prediction.

Datasets

The study uses two datasets collected from the social media platform Reddit between 2021-2022:

  • News Dataset: Consists of news articles consumed by participants from news-focused subreddits (e.g., r/worldnews, r/politics).
  • Fiction Dataset: Consists of short fiction consumed by participants from fiction-sharing subreddits (e.g., r/WritingPrompts, r/shortstories).

Participants provided their Big-5 personality inventories (BFI-2) and their public Reddit comment history was used to infer text consumption. The paper notes that while active Reddit users engage with many articles, engagement is not evenly distributed, which can introduce uncertainty in the average-personality labeling.

Key Insights and Ideas

  • Consumed text can indicate personality: This is a foundational finding, opening up a new avenue for personality inference.
  • "Message-level attention" struggles with consumed text: The assumption that individual pieces of consumed text are uniquely informative for a user's personality doesn't hold as well as it does for produced text. This suggests that the "signal" from consumed text is more diffuse.
  • Aggregate reader personality is a strong signal: The "average personality per article" method significantly outperforms message-level attention, indicating that the collective personality profile of an article's readership is a powerful predictor.
  • News content is more predictive than fiction: Models trained on news articles generally achieve better personality prediction performance than those trained on fiction.
  • Article titles can be surprisingly informative: For news articles, using only the title often yields performance comparable to using the entire article body, suggesting that concise, attention-grabbing elements are key. This also raises questions about the extent to which users actually consume the full content.

Key Results

  • Message-Level Attention: Achieved low average Pearson's r correlations, around 0.06, for predicting personality dimensions from consumed text. This is a significant drop compared to its performance on produced text.
  • Average Personality Per Article: Achieved significantly higher correlations, reaching up to r = 0.34 for predicting personality dimensions.
  • For news articles, performance was generally higher than for fiction articles.
  • For news titles, performance was comparable to using the full news article text.
  • Produced Text Performance: When applying the message-level attention model to produced text (comments) from the same participants, performance improved substantially, reaching Pearson's r around 0.32-0.33 for most dimensions, comparable to state-of-the-art. This highlights the difference in signal strength between produced and consumed text.

Limitations and Future Work

  • Consumption Certainty: The study infers consumption from commenting on Reddit threads. It's acknowledged that commenting doesn't guarantee full article reading, and the paper tentatively concludes that a condensed set of features (like titles) might be sufficient due to this uncertainty.
  • Dataset Limitations: The datasets are limited in size (around 1,100 participants) and temporal scope (2021-2022). The authors note that publicly available datasets and shared tasks would be beneficial for future research and reproduction.
  • Message Attention Adaptation: Further investigation is needed into how message-attention models can be adapted to the context of consumed text, as the current approach is not well-suited.
  • Generalizability: Behavioral research could explore if the observed patterns in personality prediction from consumed text hold outside of the Reddit platform and these specific domains.

Summary updated

BibTeX
@inproceedings{sutton2023you,
  author = {Adam Sutton and Almog Simchon and Matthew Edwards and Stephan Lewandowsky},
  title = {{You Are What You Read: Inferring Personality From Consumed Textual Content}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}

Loading related content...

Figure