PsyMem: Fine-grained psychological alignment and Explicit Memory Control for Advanced Role-Playing LLMs

Xilong Cheng, Yunxiao Qin, Yuting Tan, Zhengnan Li, Ye Wang, Hongjiang Xiao, Yuan Zhang

Abstract

Existing LLM-based role-playing methods often rely on superficial textual descriptions or simplistic metrics, inadequately modeling both intrinsic and extrinsic character dimensions. Additionally, they typically simulate character memory with implicit model knowledge or basic retrieval augment generation without explicit memory alignment, compromising memory consistency. The two issues weaken reliability of role-playing LLMs in several applications, such as trustworthy social simulation. To address these limitations, we propose PsyMem, a novel framework integrating fine-grained psychological attributes and explicit memory control for role-playing. PsyMem supplements textual descriptions with 26 psychological indicators to detailed model character. Additionally, PsyMem implements memory alignment training, explicitly trains the model to align character's response with memory, thereby enabling dynamic memory-controlled responding during inference. By training Qwen2.5-7B-Instruct on our specially designed dataset (including 5,414 characters and 38,962 dialogues extracted from novels), the resulting model, termed as PsyMem-Qwen, outperforms baseline models in role-playing, achieving the best performance in human-likeness and character fidelity.

Introduction

While Large Language Models (LLMs) have shown remarkable capabilities in role-playing (RP) for gaming, AI counseling, and social simulations, they frequently struggle with two fundamental issues. First, oversimplified characterization occurs when models rely on brief textual descriptions or narrow metrics like MBTI, failing to capture the depth of human personality and behavioral logic. Second, weak memory control leads to inconsistencies; models often rely on internal pre-trained knowledge for famous characters but struggle with original ones, even when using Retrieval-Augmented Generation (RAG), because they aren't explicitly trained to align responses with retrieved memories.

PsyMem addresses these gaps by introducing a framework that preserves persistent traits and memories across evolving social contexts, ensuring agents remain "trustworthy" and consistent.

The PsyMem Framework

The PsyMem framework introduces three core innovations to move beyond superficial role-playing:

  1. Fine-Grained Psychological Profiling: Characters are modeled using 26 quantitative indicators derived from established psychological frameworks, including the Big Five Personality traits, Schwartz’s Theory of Basic Values, Zimbardo’s Social & Leadership styles, and the Thomas-Kilmann Conflict Mode Instrument (TKI).
  2. Explicit Memory Alignment: Using GraphRAG, the framework transforms novel-based character data into knowledge graphs. Unlike standard RAG, PsyMem incorporates this memory into the training process, forcing the model to anchor its responses to specific retrieved memories.
  3. Two-Stage Training (DMT): The first stage establishes foundational role-playing using character profiles and dialogues. The second stage enhances precision by training on memory-augmented data and synthetic role-playing style data, which transforms general instruction-following data into character-specific dialogues.
Overview of the PsyMem framework including psychological attribute modeling and memory-augmented training.

Dataset Construction

To support this framework, the authors constructed a massive, high-quality dataset:

  • $D_{RP}$: Contains 38,962 dialogues involving 5,414 characters extracted from 539 novels, offering significantly more diversity than previous open-source RP datasets.
  • $D_{RP}^{synth}$: A synthetic dataset created by clustering turns from general datasets and using GPT-4o to annotate them with psychological profiles, bridging the gap between general SFT and role-playing.
  • Evaluation Set ($D_{RP}^{eval}$): To prevent data leakage, the test set uses 50 characters from novels published after June 2024.

Experimental Results

The researchers fine-tuned Qwen2.5-7B-Instruct and LLaMA3.1-8B-Instruct using the PsyMem framework. The results demonstrate that memory is more effective when treated as a training objective rather than just an input.

Performance comparison of PsyMem models against general and role-play LLMs across character fidelity and human-likeness metrics.

Key findings include:

  • Superior Fidelity: PsyMem-Qwen (7B) achieved the highest scores in Character Fidelity, surpassing much larger proprietary models like GPT-4o, Claude-3.5-Sonnet, and DeepSeek-V3.
  • Human-Likeness: In "LLM-as-a-Judge" evaluations, PsyMem-Qwen showed a massive improvement in human-likeness, scoring 87.60% compared to the base model's 64.40%.
  • Memory Accuracy: Explicit memory training allowed the models to utilize stored character information with significantly higher accuracy than standard RAG approaches.

Conclusion and Future Work

PsyMem demonstrates that fine-grained psychological alignment and explicit memory training can significantly enhance the reliability of role-playing LLMs. While effective, the authors noted a slight trade-off where memory alignment training can marginally impact character attribute alignment. Future research will explore the scalability of GraphRAG in massive simulations and investigate how these psychologically aligned models behave in long-term, multi-agent social dynamics.

Summary updated

Abstract

Existing LLM-based role-playing methods often rely on superficial textual descriptions or simplistic metrics, inadequately modeling both intrinsic and extrinsic character dimensions. Additionally, they typically simulate character memory with implicit model knowledge or basic retrieval augment generation without explicit memory alignment, compromising memory consistency. The two issues weaken reliability of role-playing LLMs in several applications, such as trustworthy social simulation. To address these limitations, we propose PsyMem, a novel framework integrating fine-grained psychological attributes and explicit memory control for role-playing. PsyMem supplements textual descriptions with 26 psychological indicators to detailed model character. Additionally, PsyMem implements memory alignment training, explicitly trains the model to align character's response with memory, thereby enabling dynamic memory-controlled responding during inference. By training Qwen2.5-7B-Instruct on our specially designed dataset (including 5,414 characters and 38,962 dialogues extracted from novels), the resulting model, termed as PsyMem-Qwen, outperforms baseline models in role-playing, achieving the best performance in human-likeness and character fidelity.
BibTeX
@misc{cheng2025psymem,
  author = {Xilong Cheng and Yunxiao Qin and Yuting Tan and Zhengnan Li and Ye Wang and Hongjiang Xiao and Yuan Zhang},
  title = {{PsyMem: Fine-grained psychological alignment and Explicit Memory Control for Advanced Role-Playing LLMs}},
  year = {2025},
  eprint = {2505.12814},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.12814},
}
Figure