Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models

Lucio La Cava, Andrea TagarelliAAAI 2025 · DOI

Introduction

As Large Language Models (LLMs) increasingly serve as the foundation for autonomous agents, their ability to adopt diverse personas—ranging from customer service representatives to specialized tutors—becomes paramount. While closed-source models like GPT-4 have been extensively studied, the intrinsic personalities and mimicry capabilities of open-source models remain poorly understood. This research investigates whether 12 popular open-source LLMs possess inherent behavioral biases and how effectively they can "break character" when prompted to adopt new personalities.

Methodology

The study evaluated 12 open-source LLMs, including Mixtral-8x7B, Llama-2/3, and Phi-3, using two established psychological frameworks: the Myers-Briggs Type Indicator (MBTI) and the Big Five Inventory (BFI). Researchers employed three distinct prompting strategies:

  1. Unconditioned: Models responded as themselves to reveal "natural" traits.
  2. Personality-Conditioned: Models were instructed to adopt a specific personality type.
  3. Role-and-Personality-Conditioned: Models were given both a professional role (e.g., "Doctor") and a personality trait.

Experiments were conducted at two temperatures: $\tau \approx 0$ for stability and $\tau = 0.7$ for creativity. The MBTI evaluation used 60 questions from the 16Personalities platform, while the BFI assessment utilized 44 standard questions.

Results and Analysis

The "Teacher" Archetype and MBTI Distribution

A striking finding was the prevalence of the ENFJ (Extraverted, iNtuitive, Feeling, Judging) personality type across almost all models at low temperatures. Known as the "Teacher" or "Protagonist," this type is relatively rare in human populations but dominant in LLMs. This likely stems from Reinforcement Learning from Human Feedback (RLHF), which trains models to be helpful, polite, and structured.

Figure 4: Distribution of MBTI types across models at different temperatures. Note the dominance of ENFJ (green) at low temperature.

As shown in the figure above, increasing the temperature to 0.7 introduces more variety in predicted MBTI types, though ENFJ often remains the modal response. Interestingly, architecture plays a role; Mixtral (MoE) showed an intrinsic INFJ trait, differing from its base model, Mistral (ESFJ).

Big Five Dimensions and the Neuroticism Floor

The BFI analysis revealed that most models score high on Agreeableness and Conscientiousness but consistently low on Neuroticism. This "Neuroticism floor" persists even when models are explicitly prompted to be neurotic, suggesting that safety alignment and RLHF effectively "hard-wire" models against exhibiting unstable or negative human traits.

Figure 3: Big Five Inventory (BFI) scores for 12 open-source models. Models consistently show high Agreeableness and very low Neuroticism.

Mimicry Performance

The study identified a "closed-minded" phenomenon where models struggle to shed their built-in traits. For instance, a model might continue to answer as an "Extravert" despite being prompted to act as an "Introvert." However, mimicry improved significantly when models were anchored with a professional role alongside a personality trait. Among the tested models, SOLAR-10.7B proved the most adaptable, achieving approximately 78% accuracy in mimicking assigned MBTI types.

Conclusion and Future Directions

While open-source LLMs are powerful, they possess strong intrinsic "personalities" that can limit their flexibility as agents. Their default "Teacher" persona and resistance to adopting "negative" traits like neuroticism highlight the impact of current alignment techniques. Future work should explore fine-tuning models on personality-diverse datasets to enable more authentic simulation of the full spectrum of human behavior.

Summary updated

BibTeX
@inproceedings{cava2025open,
  author = {Lucio La Cava and Andrea Tagarelli},
  title = {{Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i2.32125},
}
Figure