Knowledge-Enhanced Hierarchical Heterogeneous Graph for Personality Identification with Limited Training Data

Yuxuan Song, Qiudan Li, Yilin Wu, David Jingjun Xu, Daniel Dajun ZengAAAI 2025 · OpenReview

TL;DR

This paper tackles the challenge of personality identification with very limited labeled data by introducing a novel Knowledge-Enhanced Hierarchical Heterogeneous Graph (KEHHG) model. By integrating diverse external knowledge sources and leveraging both global and local graph structures, the model achieves state-of-the-art performance even when trained on a mere 1% of the available data.

What Problem Does This Paper Address?

Identifying personality traits from text is crucial for understanding user behavior, enabling applications like personalized recommendations and proactive policy prediction. However, a major hurdle is the scarcity of labeled personality data. Existing methods often rely on large datasets for training, limiting their applicability in real-world scenarios where such data is expensive and time-consuming to acquire. While some approaches use data augmentation and prompt learning, they still struggle with generalization and flexibility due to the need for extensive labeled data to learn effective distance strategies. This paper aims to bridge this gap by developing a model that can achieve high accuracy with minimal training data.

Methods

The proposed Knowledge-Enhanced Hierarchical Heterogeneous Graph (KEHHG) model has three main components:

  1. Global Multi-view Graph Node Encoding: This stage leverages the entire dataset to build global graphs. It encodes three types of external knowledge—part-of-speech (POS) tags, entities, and Linguistic Inquiry and Word Count (LIWC) categories—along with word and text representations. These are integrated into five global graphs using Graph Convolutional Networks (GCNs) to capture inherent associations and create enriched node representations. Different knowledge types are encoded using methods like BERT for word embeddings, one-hot vectors for POS and LIWC, and TransE for entities, with adjacency matrices derived from co-occurrence frequencies, PMI, or cosine similarity.
  2. Local Knowledge-Enhanced Text Representation Learning: For each individual post, a local hierarchical heterogeneous graph is constructed. This graph incorporates a "post-word-diverse knowledge" structure, connecting posts, words, and the external knowledge types learned globally. A GCN is then applied to this local graph to learn enhanced text representations that are sensitive to personality-related external information.
  3. Relation Guided Representation Optimization: This final stage refines the representations. It employs a user-level self-attention mechanism to capture intra-user relationships, strengthening the connection between posts from the same user. Additionally, a multi-label contrastive learning strategy is used to accentuate inter-label relationships, helping to distinguish between users with different personalities. This combined approach aims to learn more discriminative semantic representations.

Datasets

The experiments were conducted on three widely used datasets for personality identification:

  • PANDORA: A large-scale collection of user-generated content from Reddit, with personality labels derived from linguistic analysis and validated psychometric assessments.
  • PAN2015: A dataset from a data science competition, containing posts from social media platforms like Twitter, blogs, in English, Spanish, Dutch, and Italian. The study focused on the English data.
  • MyPersonality: A dataset sourced from a Facebook project, where users took online psychometric tests for Big Five personality traits.

The study specifically highlights its performance when trained with only 100 samples (approximately 1% of the total data set) on these datasets.

Key Insights and Ideas

  • Leveraging Diverse External Knowledge is Crucial: Integrating POS tags, entities, and LIWC categories significantly enhances personality prediction, especially under data scarcity. Removing external knowledge led to a substantial performance drop.
  • Global-Local Graph Structure is Effective: The combination of global graph encoding (learning from the entire dataset) and local graph construction (per-post) allows for both comprehensive feature learning and fine-grained representation of individual posts.
  • User-Level Relationships Matter: The user-level self-attention mechanism is vital for capturing intra-user consistency and improving performance, indicating that understanding how a user expresses themselves across multiple posts is key.
  • Contrastive Learning Aids Discrimination: Multi-label contrastive learning effectively helps differentiate between users with distinct personality traits, which is crucial for accurate classification.
  • Minimal Data is Sufficient with the Right Approach: The model demonstrates that even with a tiny fraction of labeled data (1%), state-of-the-art results can be achieved, challenging the notion that large labeled datasets are always a prerequisite.

Key Results

The proposed KEHHG model significantly outperforms state-of-the-art methods across all three datasets when trained with only 100 samples. For instance, on the PANDORA dataset, the model achieved a Macro-F1 score of 60.89%, outperforming the best baseline (Semi-PerGCN) by 1.62%. The paper emphasizes that this performance is achieved with approximately 1% of the total data, showcasing its efficiency in low-data regimes. Ablation studies confirmed the importance of each component: removing external knowledge led to a ~9.40% drop, user-level self-attention caused a ~3.48% drop, and contrastive learning removal resulted in a ~2.63% drop. The study also found that BERT embeddings performed best for word representations compared to GloVe and one-hot encodings.

Limitations and Future Work

The authors acknowledge several areas for future work:

  • Impact of Data Ratios: Further analysis is needed on how different ratios of training data affect prediction performance.
  • Computational Efficiency: For larger datasets or real-time applications, exploring graph optimization methods and strategies like offline storage and block parallel processing is suggested to address computational challenges.
  • Generalization to Other Tasks: While successful for personality identification, exploring the applicability of this KEHHG framework to other text-based classification tasks could be beneficial.

Summary updated

Abstract

Personality identification plays important roles in understanding user behavior and offering foresight ability for downstream applications. The key challenge is how to address the scarcity of labeled personality data. Recently, some studies have adopted data augmentation and prompt learning to perform personality identification. However, they still heavily require a large amount of labeled data to learn an appropriate distance strategy, which limits the generalization and flexibility of the model. This study proposes a knowledge-enhanced hierarchical heterogeneous graph model, which adopts a global multi-view graph node encoding to acquire comprehensive personality features and their inherent associations, where three types of knowledge including part-of-speech (POS) tag, entity, and Linguistic Inquiry and Word Count (LIWC) are introduced. Then, a hierarchical heterogeneous graph with a “post-word-diverse knowledge” structure is constructed for each post to obtain enhanced representation. Finally, a relation guided representation optimization that considers intra-user relationships and inter-label relationships is further developed to learn more discriminative semantic representation. Experimental results on three widely used datasets demonstrate that the model outperforms state-of-the-art methods when training with only 100 samples (approximately 1% of the total data set).
BibTeX
@inproceedings{song2025knowledge,
  author = {Yuxuan Song and Qiudan Li and Yilin Wu and David Jingjun Xu and Daniel Dajun Zeng},
  title = {{Knowledge-Enhanced Hierarchical Heterogeneous Graph for Personality Identification with Limited Training Data}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=tOQzMboa8i},
}
Figure