Modeling Personality Traits of Filipino Twitter Users

Edward Tighe, Charibeth ChengNAACL 2018

TL;DR

This paper explores the feasibility of modeling the Big Five personality traits from the Twitter activity of Filipino users, considering their unique multilingual and code-switching communication styles. They found that Conscientiousness and Extraversion are the most reliably predictable traits, and that focusing on extreme personality outliers generally improves classification accuracy.

What Problem Does This Paper Address?

The field of text-based personality recognition has made significant strides, but often focuses on a single language and overlooks the nuances of how nationality and linguistic diversity might influence communication patterns. This paper addresses this gap by investigating personality modeling for a specific demographic – Filipino Twitter users – who exhibit a rich linguistic landscape due to the prevalence of English, Filipino, and various regional languages, along with common code-switching. Understanding how these unique communication styles can be leveraged to predict personality traits is crucial for developing more inclusive and culturally sensitive AI models.

Methods

The researchers employed a multi-stage methodology:

  • Data Collection: A web application was developed to collect Twitter data and administer the Big Five Inventory (BFI) to participants.
  • Data Pre-processing: This involved tokenizing tweets using Tweetokenize, converting text to n-grams (specifically 1-grams), and applying document frequency filtering to remove overly common or rare terms.
  • Personality Trait Score Representation: To prepare scores for classification, the continuous BFI scores were discretized into various representations:
  • Continuous: Used for regression models.
  • LAH (Low, Average, High): Groups participants into three categories.
  • LH (Low, High): Groups participants into two categories.
  • LHNA (Low, High, No Average): Excludes average scorers.
  • LHNASD (Low, High, No Average, whole Standard Deviation): Excludes average scorers and uses a wider threshold for high/low.
  • Feature Extraction: Two language-independent techniques were used:
  • TFIDF (Term Frequency-Inverse Document Frequency): Captures term importance within a document and across the corpus.
  • Term Occurrence (TO): A binary representation indicating if a term is present.
  • Feature Reduction: The top 20% of features from univariate linear regression (for regression) and chi-square tests (for classification) were selected.
  • Machine Learning Algorithms: Several algorithms were experimented with, with Linear Regression, Ridge Regression, Support Vector Machines (linear SVM), and Logistic Regression highlighted for their performance.
  • Model Evaluation: Models were evaluated using Mean Squared Error (MSE), Mean Absolute Error (MAE), and R² for regression, and F₁ score and kappa statistic for classification.

Datasets

The study utilized a newly created corpus of 610,448 tweets from 250 Filipino Twitter users. Participants also completed the 44-item Big Five Inventory (BFI) for personality assessment.

Key Insights and Ideas

  • Linguistic Diversity as an Opportunity: The paper demonstrates that the multilingual and code-switching nature of Filipino communication on Twitter can be a valuable source of information for personality modeling, rather than a barrier.
  • Conscientiousness and Extraversion are "Easier" to Model: These traits consistently showed better performance in both regression and classification across various configurations.
  • The Power of Outliers: Focusing on participants with extreme personality scores (high or low) generally led to better classification performance, especially for Conscientiousness, Extraversion, and Agreeableness. This suggests that distinct communication patterns are more pronounced at the extremes of these traits.
  • Openness is a Challenge: This trait proved to be the most difficult to model, particularly in classification, suggesting that its expression in text might be more subtle or require a broader range of features.
  • Feature Representation Matters: The choice of how to represent personality scores (continuous vs. discretized, and which discretization method) significantly impacts model performance.

Key Results

  • Best Performing Traits: Conscientiousness achieved the highest R² (0.1523) in regression and kappa (0.5516) in classification. Extraversion followed with R² of 0.1035 and kappa of 0.4376.
  • Classification Performance: Models generally benefited from using the LHNASD (Low, High, No Average, whole Standard Deviation) representation for Conscientiousness, Extraversion, and Agreeableness.
  • Regression Performance: Regression models showed generally low performance, but TFIDF features provided some predictive power for Conscientiousness and Extraversion.
  • Language Use: Approximately 58.14% of tweets were in English, 31.89% in Tagalog, with the remainder in other languages or undefined. The models retained tweets in all languages.

Limitations and Future Work

  • Outlier Removal for Neuroticism and Openness: The study found that removing extreme outliers (±1SD) did not benefit models for Neuroticism and Openness, suggesting that these traits might not have easily distinguishable patterns among extreme scorers.
  • Limited Regression Performance: The regression models did not yield strong results, indicating that current features might not fully capture the continuous nature of personality traits.
  • Future Directions: The authors suggest further analysis of content to understand how personality traits manifest in Filipino communication. They also propose exploring more advanced feature extraction techniques (e.g., topic modeling) and integrating multiple language-specific resources. Additionally, creating models for specific demographic groups (age, gender, nationality) is recommended for a deeper understanding of linguistic styles.

Summary updated

BibTeX
@inproceedings{tighe2018modeling,
  author = {Edward Tighe and Charibeth Cheng},
  title = {{Modeling Personality Traits of Filipino Twitter Users}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
}
Figure