TL;DR
This paper introduces ProtoMBTI, a novel framework that leverages prototype theory to improve personality inference from text, specifically for the MBTI (Myers-Briggs Type Indicator) framework. By aligning AI reasoning with human cognitive processes, ProtoMBTI achieves state-of-the-art accuracy and offers more interpretable and generalizable personality predictions.
What Problem Does This Paper Address?
Inferring personality from text, particularly for frameworks like MBTI, has traditionally been treated as a rigid classification problem. This approach overlooks the nuanced, graded nature of human personality judgments, where individuals don't perfectly fit into discrete categories but rather exhibit varying degrees of similarity to prototypical examples. Existing methods often struggle with this inherent fuzziness, leading to limitations in accuracy, interpretability, and generalization, especially when dealing with diverse language styles and subtle personality expressions. The authors aim to bridge this gap by developing a system that mimics human cognitive processes of categorization based on prototypes.
Methods
ProtoMBTI introduces a "cognitively aligned" framework that operationalizes prototype theory within a Large Language Model (LLM) pipeline. The core innovation lies in its retrieve-reuse-revise-retain cycle for inference.
- Prototype Bank Construction:
- LLM-Guided Data Augmentation: They use LLMs (like GPT-4o) to generate synthetic text samples that are semantically, linguistically, and sentimentally augmented. This process is guided by specific prompts for each MBTI type to ensure stylistic fidelity and class balance. A 4D Classifier acts as a quality filter to ensure generated samples are accurate.
- LoRA-Fine-tuned Encoder: A lightweight encoder (under 2B parameters) is fine-tuned using LoRA on the augmented dataset. This encoder learns to generate discriminative embeddings that capture the relationship between text and personality prototypes. These prototypes are stored as triplets:
(text_attribute, embedding, category). - Prototype-Based Reasoning (Inference):
- Retrieve: For a given input post, the system retrieves the top-k most similar prototypes from the bank based on embedding similarity.
- Reuse: The retrieved prototypes provide evidence for the prediction.
- Revise: The model aggregates evidence through prompt-based voting and revises predictions when inconsistencies arise.
- Retain: If a prediction is correct, the sample is retained to continually enrich the prototype library, enabling adaptive learning.
Datasets
The study utilizes two standard MBTI datasets:
- Kaggle: 8,675 samples from PersonalityCafe.
- Pandora: 9,067 samples from Reddit.
These datasets were preprocessed and augmented. The test sets remained unaugmented to provide a realistic evaluation of generalization under distribution shift.
Key Insights and Ideas
- Prototype Theory for Personality: The paper strongly advocates for applying prototype theory to personality modeling, arguing it better reflects human cognition than rigid classification.
- Cognitive Alignment: Aligning AI reasoning with psychological principles (like prototype theory) leads to improved performance, interpretability, and transferability.
- Retrieve-Reuse-Revise-Retain Cycle: This novel inference mechanism, inspired by Case-Based Reasoning but adapted for prototypes, is key to ProtoMBTI's success.
- LLM-Augmented Prototypes: LLMs are crucial for generating diverse, high-quality, and cognitively aligned prototypes, overcoming data imbalance and scarcity.
- The Importance of Prototype Selection: Using random or non-typical prototypes can actually harm performance, highlighting the need for carefully constructed and representative prototypes.
- Fine-grained vs. Coarse-grained: The 16-type MBTI classification is significantly more challenging than the four dichotomies, and ProtoMBTI shows its strength in handling this fine-grained task.
Key Results
- State-of-the-Art Performance: ProtoMBTI significantly outperforms existing baselines on both Kaggle and Pandora datasets for both the four MBTI dichotomies and the full 16-type classification.
- On Kaggle, it achieves an average accuracy of 85.14% across the four dimensions, a 7.35% improvement over prior work.
- For the 16-type task, ProtoMBTIQwen achieves 71.42% accuracy on Kaggle, a remarkable improvement over the best prior theoretical value of 35.89%.
- Robust Cross-Dataset Generalization: ProtoMBTI exhibits superior transferability, achieving 96.41% average accuracy on the Pandora test set when trained on mixed data, a 30.64% increase over previous results.
- Interpretability: The prototype-based reasoning provides case-based rationales, making the model's decisions more understandable.
- Efficiency: The framework uses a lightweight encoder and LLM inference, offering these benefits at a fraction of the compute cost of very large LLMs.
Limitations and Future Work
- LLM Dependence: While the framework is backbone-agnostic, outcomes may vary across different LLMs used for augmentation and inference. The authors fixed prompt templates for reproducibility.
- Multi-class Baselines: While ProtoMBTI outperforms existing methods, future work could explore direct multi-class baselines for comparison.
- Broader Validation: Further validation across more diverse datasets and domains is recommended.
- Extending to Other Models: The prototype-driven reasoning paradigm is suggested to be extendable to other soft-label personality models (e.g., Big Five), sentiment analysis, and multimodal classification.
- Computational Cognitive Science: Future work could incorporate more recent research in computational cognitive science to further refine the theoretical grounding.
Summary updated