Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing

Letian Peng, Jingbo Shang

Introduction

Persona-driven role-playing (PRP) is a specialized application of Large Language Models (LLMs) where the model is tasked with adopting a specific identity based on a set of background facts. While modern LLMs demonstrate impressive mimicry, maintaining "global faithfulness"—the ability to consistently adhere to a persona without hallucinating contradictory details or ignoring relevant facts—remains a significant challenge.

Current evaluation methods for PRP often rely on "black box" scoring, where a powerful model like GPT-4 provides a coarse numerical rating (e.g., 1-10). This approach lacks transparency, is computationally expensive at scale, and offers little diagnostic information for model improvement. This paper introduces the Active-Passive-Constraint (APC) framework, a granular approach to quantifying and optimizing persona faithfulness by treating role-playing as a Constraint Satisfaction Problem (CSP).

The APC Framework: Role-playing as Constraint Satisfaction

The core innovation of the APC framework lies in decomposing a persona into individual atomic statements and categorizing them based on their relevance to a specific user query. This creates two distinct types of constraints:

  1. Active Constraints: These are persona statements directly relevant to the user's query. To remain faithful, the LLM's response must explicitly entail or agree with these statements.
  2. Passive Constraints: These are statements irrelevant to the current query. While the model does not need to mention them, it must remain "neutral" and avoid generating content that contradicts them.

By distinguishing between these two, the APC framework ensures that a character doesn't just answer questions correctly but also maintains a consistent background identity throughout the conversation.

Quantifying Faithfulness: The APC Score

To measure adherence to these constraints, the researchers developed the APC Score. This metric utilizes Natural Language Inference (NLI) to calculate the probabilities of Entailment, Neutrality, and Contradiction for every persona statement relative to the model's response.

Because evaluating hundreds of statements per response using a large model like GPT-4 is prohibitively slow, the authors employed Symbolic Distillation. They used GPT-4 to generate a labeled dataset of relevance and NLI relationships, which was then used to fine-tune a smaller, more efficient "student" model (DeBERTa-V3, ~300M parameters). This distilled discriminator achieved 90% logic consistency with GPT-4 while operating approximately six times faster than a 2B parameter LLM, making large-scale evaluation feasible.

Optimization via Direct Preference Optimization (DPO)

Beyond evaluation, the APC framework provides a mechanism for model improvement. The authors used the APC score as a reward signal for Direct Preference Optimization (DPO). By training on pairs of responses where one is more faithful to the persona than the other (as determined by the APC score), the model learns the general principle of constraint satisfaction. This "global" optimization allows the model to improve its faithfulness across all persona statements, rather than just memorizing specific answers.

Experimental Results

The framework was tested using both handcrafted characters (to avoid data contamination) and famous historical or fictional figures with extensive persona documents (up to 600 statements). Key findings include:

  • Human Alignment: The APC score demonstrated a high correlation with human judgments of role-playing quality, validating it as a robust proxy for faithfulness.
  • RAG vs. Long-Context: The study confirmed the "lost-in-the-middle" phenomenon. As persona length increases, Retrieval-Augmented Generation (RAG) typically outperforms Long-Context Memory, as the latter struggles to manage hundreds of simultaneous constraints.
  • Performance Gains: Applying APC-based DPO significantly improved faithfulness scores across various models. For instance, the character "Eve" (a secretive spy) saw a marked increase in the APC score when RAG was combined with DPO.
  • Scalability: The system remained effective even for complex personas like Martin Luther King Jr., demonstrating its utility for high-density character profiles.

Discussion and Limitations

While the APC framework represents a significant step forward, several areas for improvement remain. Currently, the framework treats all persona statements as equally important. Future iterations could incorporate weighted constraints, recognizing that core identity traits (e.g., a character's name) are more critical than minor details. Additionally, while the distilled DeBERTa model is efficient, the requirement to check every persona statement for every response still introduces computational overhead. Finally, as the discriminators are distilled from GPT-4, they may inherit some of the teacher model's inherent biases.

In conclusion, the APC framework provides a transparent, efficient, and optimizable path toward creating AI characters that are truly faithful to their assigned personas, moving beyond simple mimicry toward consistent, constraint-aware role-playing.

Summary updated

Abstract

Persona-driven role-playing (PRP) aims to build AI characters that can respond to user queries by faithfully sticking with \emph{all} (factual) statements in persona documents. Unfortunately, existing faithfulness criteria for PRP are limited to coarse-grained LLM-based scoring without a clear definition or formulation. This paper presents a pioneering exploration to quantify PRP faithfulness evaluation as a fine-grained and explainable criterion, which also serves as a reliable reference for faithfulness optimization. Our criterion first discriminates persona statements into \emph{active} and \emph{passive} constraints by identifying the query-statement relevance. Then, we incorporate all constraints following the principle that the AI character's response should be (a) entailed by active constraints and (b) not contradicted by passive constraints. We translate this principle mathematically into a novel Active-Passive-Constraint (APC) score, a constraint-wise sum of statement-to-response natural language inference (NLI) scores weighted by constraint-query relevance scores. In practice, we build the APC scoring system by symbolically distilling small NLI and relevance discriminators (300M parameters) from GPT-4 for efficiency, and both show high consistency with GPT-4's discrimination. We validate the quality of the APC score against human evaluation based on example personas with tens of statements, and the results show a high correlation. As the APC score could faithfully reflect the PRP quality, we further leverage it as a reward system in direct preference optimization (DPO) for better AI characters. Our experiments offer a fine-grained and explainable comparison between existing PRP techniques, revealing their advantages and limitations. We further find APC-based DPO to be one of the most competitive techniques for sticking with all constraints and can be well incorporated with other techniques. We then extend the scale of the experiments to real persons with hundreds of statements and reach a consistent conclusion. Finally, we provide comprehensive analyses and case studies to support the effectiveness of APC and APC-based DPO.
BibTeX
@inproceedings{peng2024quantifying,
  author = {Letian Peng and Jingbo Shang},
  title = {{Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=bzPmjmiaz8},
}
Figure