TL;DR
This paper introduces "personality vectors," a novel method for imbuing large language models (LLMs) with specific personality traits by merging pre-trained model weights. This approach allows for fine-grained control over trait intensity and the composition of multiple traits without requiring additional training, demonstrating promising transferability across different model types.
What Problem Does This Paper Address?
The increasing demand for personalized AI agents has spurred research into aligning LLM behavior with human personality traits. While existing methods can induce personality, they often struggle to capture the continuous and multidimensional nature of human personality. Current approaches lack fine-grained control over trait intensity and the ability to seamlessly combine multiple traits, limiting the development of truly personalized LLMs.
Methods
The core innovation is the creation of "personality vectors." These vectors are derived by subtracting the weights of a general pre-trained LLM from the weights of a model fine-tuned on a specific personality trait (e.g., high extraversion). This difference captures the "personality signature." These vectors can then be merged into other LLMs using model merging techniques, such as task arithmetic, TIES-Merging, and DaRE, to imbue them with the desired personality. The intensity of the personality can be controlled by scaling the personality vector during merging.
Datasets
The primary dataset used for fine-tuning personality-specific models was the Big5-Chat dataset. This dataset is structured around the Big Five personality framework and contains 100,000 dialogue examples, with 10,000 examples for each of the 10 personality categories (high/low levels of Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism).
For evaluation, the authors used:
- Big Five Inventory (BFI) questionnaire: Used to assess personality traits through model-generated responses.
- LIWC-22: For analyzing linguistic features in generated text.
- AlpacaEval: To evaluate instruction-following capabilities.
- PsychoFlicker dataset: Used for evaluating Vision-Language Models (VLMs) and their interpretation of images based on personality.
Key Insights and Ideas
- Personality as a Vector: The paper demonstrates that personality traits can be effectively represented as vectors in the model's weight space, allowing for manipulation through vector arithmetic.
- Fine-Grained Control: Scaling personality vectors provides a continuous knob to adjust the intensity of a personality trait, going beyond binary high/low settings.
- Compositionality of Traits: Multiple personality vectors can be merged to create models with complex, multi-faceted personalities, mimicking the nuanced nature of human traits.
- Transferability is Key: Personality vectors are not tied to specific tasks or domains; they can be effectively transferred to different model architectures (e.g., RPAs, multilingual models, VLMs), suggesting they capture fundamental aspects of personality representation.
- Implicit vs. Explicit Steering: Personality vectors seem to act as "steering signals" that highlight existing trait-relevant features in LLMs, rather than creating them from scratch, which is more robust than prompt-based methods.
Key Results
- Personality Scaling: Personality vector merging showed a strong positive correlation (>0.9) between the scaling coefficient and the resulting BFI score for the associated trait, demonstrating effective control over trait intensity.
- Multi-Trait Composition: Merging multiple personality vectors, especially with DaRE for interference mitigation, significantly improved trait correlation compared to merging without it.
- Personality Negation: Subtracting a personality vector (negative scaling) could induce opposite traits, though it sometimes led to a reduction in general conversational ability.
- Cross-Domain Transfer: Personality vectors successfully modulated the personalities of Role-Playing Agents (RPAs), Korean language models, and Vision-Language Models (VLMs), indicating broad applicability. For instance, a Beethoven RPA's low Extraversion score increased significantly after merging with a high Extraversion vector.
- VLM Alignment: Merged VLMs showed a clear preference for images that matched their assigned personality traits, demonstrating personality's influence on visual-linguistic reasoning.
Limitations and Future Work
- Parameter Interference: Personality vectors trained on dialogue datasets can share overlapping parameter spaces, leading to interference during multi-vector merging. While DaRE helps, future work could explore orthogonal vector construction or specialized merging algorithms.
- Exploration Scope: The study did not exhaustively tune all merging parameters (e.g., scaling coefficients for each trait in multi-trait merging) and suggests future work could benefit from automated parameter tuning.
- Ethical Considerations: The authors acknowledge the potential for misuse, noting that modulating traits like lowering Agreeableness or raising Neuroticism can lead to aggressive tones. They emphasize the need for caution and oversight when applying these techniques.
- Future Work: The authors suggest exploring more robust solutions for parameter interference, developing automated methods for tuning merging parameters, and further investigating the ethical implications of personality modulation.
Summary updated