TL;DR
This paper tackles the challenge of personality identification with very limited labeled data by introducing a novel Knowledge-Enhanced Hierarchical Heterogeneous Graph (KEHHG) model. By integrating diverse external knowledge sources and leveraging both global and local graph structures, the model achieves state-of-the-art performance even when trained on a mere 1% of the available data.
What Problem Does This Paper Address?
Identifying personality traits from text is crucial for understanding user behavior, enabling applications like personalized recommendations and proactive policy prediction. However, a major hurdle is the scarcity of labeled personality data. Existing methods often rely on large datasets for training, limiting their applicability in real-world scenarios where such data is expensive and time-consuming to acquire. While some approaches use data augmentation and prompt learning, they still struggle with generalization and flexibility due to the need for extensive labeled data to learn effective distance strategies. This paper aims to bridge this gap by developing a model that can achieve high accuracy with minimal training data.
Methods
The proposed Knowledge-Enhanced Hierarchical Heterogeneous Graph (KEHHG) model has three main components:
- Global Multi-view Graph Node Encoding: This stage leverages the entire dataset to build global graphs. It encodes three types of external knowledge—part-of-speech (POS) tags, entities, and Linguistic Inquiry and Word Count (LIWC) categories—along with word and text representations. These are integrated into five global graphs using Graph Convolutional Networks (GCNs) to capture inherent associations and create enriched node representations. Different knowledge types are encoded using methods like BERT for word embeddings, one-hot vectors for POS and LIWC, and TransE for entities, with adjacency matrices derived from co-occurrence frequencies, PMI, or cosine similarity.
- Local Knowledge-Enhanced Text Representation Learning: For each individual post, a local hierarchical heterogeneous graph is constructed. This graph incorporates a "post-word-diverse knowledge" structure, connecting posts, words, and the external knowledge types learned globally. A GCN is then applied to this local graph to learn enhanced text representations that are sensitive to personality-related external information.
- Relation Guided Representation Optimization: This final stage refines the representations. It employs a user-level self-attention mechanism to capture intra-user relationships, strengthening the connection between posts from the same user. Additionally, a multi-label contrastive learning strategy is used to accentuate inter-label relationships, helping to distinguish between users with different personalities. This combined approach aims to learn more discriminative semantic representations.
Datasets
The experiments were conducted on three widely used datasets for personality identification:
- PANDORA: A large-scale collection of user-generated content from Reddit, with personality labels derived from linguistic analysis and validated psychometric assessments.
- PAN2015: A dataset from a data science competition, containing posts from social media platforms like Twitter, blogs, in English, Spanish, Dutch, and Italian. The study focused on the English data.
- MyPersonality: A dataset sourced from a Facebook project, where users took online psychometric tests for Big Five personality traits.
The study specifically highlights its performance when trained with only 100 samples (approximately 1% of the total data set) on these datasets.
Key Insights and Ideas
- Leveraging Diverse External Knowledge is Crucial: Integrating POS tags, entities, and LIWC categories significantly enhances personality prediction, especially under data scarcity. Removing external knowledge led to a substantial performance drop.
- Global-Local Graph Structure is Effective: The combination of global graph encoding (learning from the entire dataset) and local graph construction (per-post) allows for both comprehensive feature learning and fine-grained representation of individual posts.
- User-Level Relationships Matter: The user-level self-attention mechanism is vital for capturing intra-user consistency and improving performance, indicating that understanding how a user expresses themselves across multiple posts is key.
- Contrastive Learning Aids Discrimination: Multi-label contrastive learning effectively helps differentiate between users with distinct personality traits, which is crucial for accurate classification.
- Minimal Data is Sufficient with the Right Approach: The model demonstrates that even with a tiny fraction of labeled data (1%), state-of-the-art results can be achieved, challenging the notion that large labeled datasets are always a prerequisite.
Key Results
The proposed KEHHG model significantly outperforms state-of-the-art methods across all three datasets when trained with only 100 samples. For instance, on the PANDORA dataset, the model achieved a Macro-F1 score of 60.89%, outperforming the best baseline (Semi-PerGCN) by 1.62%. The paper emphasizes that this performance is achieved with approximately 1% of the total data, showcasing its efficiency in low-data regimes. Ablation studies confirmed the importance of each component: removing external knowledge led to a ~9.40% drop, user-level self-attention caused a ~3.48% drop, and contrastive learning removal resulted in a ~2.63% drop. The study also found that BERT embeddings performed best for word representations compared to GloVe and one-hot encodings.
Limitations and Future Work
The authors acknowledge several areas for future work:
- Impact of Data Ratios: Further analysis is needed on how different ratios of training data affect prediction performance.
- Computational Efficiency: For larger datasets or real-time applications, exploring graph optimization methods and strategies like offline storage and block parallel processing is suggested to address computational challenges.
- Generalization to Other Tasks: While successful for personality identification, exploring the applicability of this KEHHG framework to other text-based classification tasks could be beneficial.
Summary updated