The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agents

Yifan Duan, Yihong Tang, Xuefeng Bai, Kehai Chen, Juntao Li, Min Zhang

Abstract

Large language models (LLMs) excel in both closed tasks (including problem-solving and code generation) and open tasks (including creative writing), yet existing explanations for their capabilities lack connections to real-world human intelligence. To fill this gap, this paper systematically investigates LLM intelligence through the lens of "human simulation," addressing three core questions: (1) How do personality traits affect problem-solving in closed tasks? (2) How do traits shape creativity in open tasks? (3) How does single-agent performance influence multi-agent collaboration? By assigning Big Five personality traits to LLM agents and evaluating their performance in single- and multi-agent settings, we reveal that specific traits significantly influence reasoning accuracy (closed tasks) and creative output (open tasks). Furthermore, multi-agent systems exhibit collective intelligence distinct from individual capabilities, driven by distinguishing combinations of personalities. We demonstrate that LLMs inherently simulate human behavior through next-token prediction, mirroring human language, decision-making, and collaborative dynamics.

Introduction

While Large Language Models (LLMs) have demonstrated remarkable proficiency across a spectrum of tasks—from complex reasoning to creative synthesis—the underlying mechanisms of these "emergent" abilities remain poorly understood in the context of human psychology. Previous research has touched upon "role-playing" as a method to enhance performance, but a systematic exploration of how core psychological traits influence model behavior has been missing. This study addresses this gap by investigating LLM intelligence through the lens of "human simulation," specifically focusing on the Big Five personality traits: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.

Methodology: Human Simulation on Psychology

The researchers developed a comprehensive framework to simulate and validate human-like personality traits within LLM agents. The process follows a multi-step pipeline designed to shape, verify, and test these simulated personalities.

Figure 1: The systematic framework for Human Simulation on Psychology, detailing the steps from personality shaping to multi-agent evaluation.

Personality Shaping and Validation

The framework begins with Personality Shaping, where agents are "nudged" into specific profiles using prompts derived from established psychological scales. To ensure the validity of these simulations, the agents are administered the Big Five Inventory-2 (BFI-2). The study tested 32 distinct personality combinations across three prominent models: Qwen2.5-14B-Instruct, Qwen2.5-32B-Instruct, and Llama3.1-8B-Instruct.

Evaluation Framework

The agents were evaluated in both single-agent and multi-agent settings:

  • Closed Tasks: Focused on objective reasoning and knowledge using benchmarks like MMLU, ARC, and SciQ.
  • Open Tasks: Focused on creativity using the Torrance Tests of Creative Thinking (TTCT) criteria, including tasks like the Alternative Uses Task (AUT).
  • Multi-Agent Collaboration: Employed an LLM debate framework where agents with diverse personalities discussed problems to reach a consensus.

Experimental Results

Personality Plasticity and Reasoning

The study found that LLMs exhibit significant "personality plasticity," successfully adopting and maintaining profiles that are statistically detectable via standard psychological tools. Larger models, such as Qwen-32B, demonstrated a higher capacity for this shaping.

In closed-task reasoning, personality traits showed clear correlations with performance. Notably, Neuroticism acted as a "tax" on performance, consistently correlating with lower accuracy across benchmarks. Conversely, Openness and Conscientiousness generally bolstered reasoning capabilities.

Figure 3: Correlation heatmaps showing the relationship between Big Five personality traits and performance on closed-task benchmarks across three different LLMs.

Creativity and Open-Ended Tasks

The strongest positive correlation observed was between Openness to Experience and creativity scores, particularly in Originality and Elaboration. This mirrors human psychological findings where openness is a primary driver of creative output. Conversely, high Neuroticism inhibited creative processing, similar to the effects of anxiety in humans.

Multi-Agent Dynamics

The research highlighted that multi-agent systems possess a "collective intelligence" that transcends the sum of individual parts. Teams with high personality diversity often outperformed homogeneous teams on complex tasks, suggesting that varying perspectives—driven by personality—can enhance group problem-solving. However, traits like low Agreeableness or high Neuroticism in team members often led to friction and reduced collaborative effectiveness.

Conclusion

This investigation demonstrates that LLMs can effectively simulate human personality traits, which in turn significantly influence their cognitive and creative performance. By mirroring human psychological patterns, LLMs show that their capabilities are not just statistical artifacts but can be understood through the framework of human simulation. Future research will focus on optimizing team formations and exploring the continuous spectrum of personality traits to further enhance LLM utility.

Summary updated

Abstract

Large language models (LLMs) excel in both closed tasks (including problem-solving, and code generation) and open tasks (including creative writing), yet existing explanations for their capabilities lack connections to real-world human intelligence. To fill this gap, this paper systematically investigates LLM intelligence through the lens of ``human simulation'', addressing three core questions: (1) How do personality traits affect problem-solving in closed tasks? (2) How do traits shape creativity in open tasks? (3) How does single-agent performance influence multi-agent collaboration? By assigning Big Five personality traits to LLM agents and evaluating their performance in single- and multi-agent settings, we reveal that specific traits significantly influence reasoning accuracy (closed tasks) and creative output (open tasks). Furthermore, multi-agent systems exhibit collective intelligence distinct from individual capabilities, driven by distinguishing combinations of personalities. We demonstrate that LLMs inherently simulate human behavior through next-token prediction, mirroring human language, decision-making, and collaborative dynamics.
BibTeX
@misc{duan2025power,
  author = {Yifan Duan and Yihong Tang and Xuefeng Bai and Kehai Chen and Juntao Li and Min Zhang},
  title = {{The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agents}},
  year = {2025},
  eprint = {2502.20859},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.20859},
}
Figure