Automatic Recognition of Personality in Conversation

François Mairesse, Marilyn WalkerNAACL 2006

TL;DR

This paper explores automatically identifying personality traits from conversational speech. They developed models that can predict Big Five personality dimensions from linguistic and prosodic features, outperforming random baselines and revealing new linguistic markers of personality. This work is a crucial step towards building more adaptive dialogue systems and understanding human interaction.

What Problem Does This Paper Address?

Understanding personality from how people speak is a long-standing area of interest with applications ranging from leader identification in meetings to improving online dating matches. While it's intuitively understood that our language reveals aspects of our personality, automatically quantifying this relationship has been challenging. Previous work has often focused on specific linguistic features or acted speech. This paper tackles the problem of building robust, generalizable models that can automatically recognize the "Big Five" personality dimensions (Extraversion, Emotional Stability, Agreeableness, Conscientiousness, and Intellect) from real-world, unscripted conversations of unseen individuals.

Methods

The authors employed a ranking-based approach using RankBoost, a machine learning algorithm designed for learning ranking functions. Instead of predicting an absolute personality score, their models learn to compare two conversation extracts and determine which one exhibits a higher degree of a specific personality trait. This "pairwise comparison" approach is well-suited for personality assessment, which is often relative.

They extracted a comprehensive set of features from the conversational data, categorized as:

  • LIWC features: These are standard linguistic categories from the Linguistic Inquiry and Word Count (LIWC) tool, covering aspects like pronoun usage, emotional processes, cognitive processes, and social concerns.
  • MRC features: These are psycholinguistic features derived from the MRC Psycholinguistic Database, including word length, frequency, familiarity, and concreteness.
  • Utterance type features: Heuristics were used to classify utterances into types like commands, prompts, questions, and assertions, aiming to capture conversational initiative.
  • Prosodic features: Using the Praat software, they extracted acoustic features such as average, minimum, maximum, and standard deviation of pitch and intensity, as well as measures of speech rate and voiced time.

Models were trained using these features and evaluated using a ranking error function, which measures the percentage of misordered pairs.

Datasets

The primary dataset used in this study was collected by Mehl et al. (in press). It consists of daily-life conversation extracts from 96 participants. These extracts were recorded using an Electronically Activated Recorder (EAR) over two days. The corpus contains approximately 97,468 words and 15,269 utterances. Personality ratings for the participants were obtained from 5-7 independent observers who scored each extract using the Big Five Inventory (John and Srivastava, 1999). The authors report strong inter-observer reliability for these ratings.

Key Insights and Ideas

  • Personality is detectable in everyday speech: The core finding is that automatic analysis of conversational language can indeed reveal personality traits, even for unseen individuals.
  • Prosody matters, especially for Extraversion: Prosodic features (pitch, intensity, speech rate) were surprisingly strong predictors of Extraversion, even outperforming the full feature set in some cases. This suggests that how someone speaks is as important as what they say for this trait.
  • Linguistic markers confirm and extend prior findings: The models confirmed existing knowledge (e.g., extraverts talk more, neurotics use more first-person pronouns) but also uncovered many novel linguistic and prosodic markers.
  • Different features excel for different traits: No single feature set dominated across all personality dimensions. LIWC features were generally strong, but MRC features were particularly good for predicting Emotional Stability.
  • Ranking is a powerful paradigm: The pairwise ranking approach proved effective, allowing models to learn subtle differences in personality expression.

Key Results

  • Significant improvement over baseline: All models trained on the full feature set showed statistically significant improvements over a random ordering baseline for Extraversion, Agreeableness, Conscientiousness, and Intellect.
  • Best performance for Agreeableness and Conscientiousness: These traits achieved the lowest ranking errors (0.31 and 0.33 respectively).
  • Prosody's dominance in Extraversion: Prosodic features alone yielded a lower ranking error (0.26) for Extraversion than the full feature set.
  • Utterance type features were less effective: On their own, utterance type features did not outperform the baseline.
  • Emotional Stability is challenging: This trait was the most difficult to model, with models performing less effectively compared to other dimensions.

Limitations and Future Work

  • Data limitations: The authors acknowledge that their dataset, while substantial, is relatively small (96 subjects). They anticipate that more training data would improve model accuracies.
  • Speech recognition noise: For applications involving speech recognition, recognition errors can introduce noise into all features except prosodic ones.
  • Future directions: The authors plan to integrate these personality recognition models into dialogue systems to enable adaptive language generation. They also aim to explore methods for further improving accuracy.

Summary updated

BibTeX
@inproceedings{mairesse2006automatic,
  author = {François Mairesse and Marilyn Walker},
  title = {{Automatic Recognition of Personality in Conversation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2006},
}
Figure