Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness

Hyunwoo Kim, Byeongchang Kim, Gunhee Kim

TL;DR

State-of-the-art dialogue agents frequently contradict their assigned personas because standard decoding objectives prioritize local language fluency over long-term consistency. This paper introduces a plug-and-play framework that endows agents with "public self-consciousness" via an imaginary listener, drastically reducing contradictions on-the-fly without requiring expensive Natural Language Inference (NLI) labels or extra training modules.

What Problem Does This Paper Address?

Endowing open-domain dialogue agents with a persistent "persona" (e.g., "I am a construction worker who loves cats") helps them generate more engaging responses. However, even advanced generative models like Blender or GPT-based architectures remain highly insensitive to contradictions. An agent might say "I like going outside" and follow it up immediately with "I like to stay at home."

Figure 1: Illustration of the consistency issue in dia- logue. While a literal dialogue agent (S0) fails to de- liver a consistent persona, our self-conscious agent (S1) does so, by modeling an imaginary listener. Icons are designed by Nhor Phai and Vincent Le Moign.

Existing methods fix this by training separate Natural Language Inference (NLI) models to classify pairs of utterances as entailment, neutral, or contradiction, using these scores to rerank outputs. However, this supervised approach is data-hungry, computationally expensive, and generalizes poorly due to dataset-specific distribution shifts. This paper addresses the challenge of maintaining persona consistency without relying on NLI labels or modifying the underlying generative model's parameters.

Methods

The authors draw inspiration from social cognition: humans maintain consistency by predicting how they are perceived by others—a trait known as public self-consciousness. To mimic this, they adapt the Rational Speech Acts (RSA) framework, treating token generation as a recursive game between an agent and an imaginary listener.

Figure 3: The proposed self-conscious agent S1 con- sists of base speaker S0 and imaginary listener L0. It recursively generates the next token ut at every time t.

The system operates token-by-token during decoding:

  1. Base Speaker ($S_0$): A standard pre-trained conditional dialogue model provides a baseline probability distribution for the next token.
  2. Imaginary Listener ($L_0$): Using Bayes' rule, the listener evaluates how likely a candidate token is to help them correctly guess the speaker's true persona out of a small set of choices (the true persona plus "distractor" personas).
  3. Self-Conscious Speaker ($S_1$): The final token distribution combines the base speaker's fluency with the listener's feedback. Tokens that make the correct persona obvious receive a probability bonus.

Learnable Distractor Selection

A key novelty is how the framework selects the alternative "distractor" personas that the listener uses to test the speaker. Rather than choosing them randomly, the authors implement a Distractor Memory (DM) module using a life-long memory network. It uses a fine-tuned BERT query network to retrieve personas from the training set that are contextually relevant but distinct, forcing the agent to generate highly specific, non-contradictory language.

Datasets

The approach was evaluated using three primary datasets:

  • Dialogue NLI: An evaluation set of 542 dialogue instances explicitly annotated with entailing, neutral, and contradictory utterance candidates to test ranking performance.
  • PersonaChat: A benchmark dataset containing 17,878 training and 1,000 validation conversations conditioned on explicit persona profiles.
  • EmpatheticDialogues: Used to evaluate the model's ability to generalize to context-consistency (staying consistent with dialogue history rather than a fixed persona profile).

Key Insights and Ideas

  • Language Models are Blind to Persona Context: Perplexity analysis revealed that standard models exhibit significantly lower perplexity for contradictory words than for the correct persona words. They behave like plain language models, prioritizing local syntactic patterns over global persona constraints.
  • Dissimilar Distractors Work Best: When training the Distractor Memory module, selecting distractors that were semantically farthest from the target persona yielded better consistency than selecting nearest neighbors. Contrast helps the imaginary listener better adjudicate the agent's identity.
  • Dynamic Prior Updates Matter: Instead of using a static uniform prior over personas at every step, updating the listener's world prior dynamically based on previously generated tokens preserves cumulative context and prevents the model from drifting into contradictions mid-sentence.

Key Results

The self-conscious framework ($S_1$ + DM) was evaluated across three strong base models: ControlSeq2Seq, TransferTransfo, and Blender.

Quantitative results on the Dialogue NLI evaluation set.
  • Contradiction Reduction: On the Dialogue NLI dataset, the proposed method cut contradiction rates (Contradict@1) by more than half for all models. For instance, Blender's contradiction rate dropped from 42.4% to 19.6%.
  • Accuracy Boost: The accuracy of selecting the ground-truth utterance (Hits@1) improved significantly across the board (e.g., from 11.1% to 18.8% for TransferTransfo).
  • Outperforming NLI Baselines: Remarkably, the unsupervised $S_1$ + DM agent outperformed standard base models that used fully supervised, explicitly trained NLI rerankers on the Hits@1 metric.
  • Human Evaluation: Human judges on Amazon Mechanical Turk rated the self-conscious agent as significantly more consistent than the baseline models while maintaining an equivalent level of conversational engagingness.

Limitations and Future Work

While highly effective, the framework currently relies on manual grid searches to tune the speaker and listener rationality coefficients ($\alpha$ and $\beta$). If these values are set too high, the language model degenerates, simply copying fragments of the persona text verbatim rather than generating natural dialogue.

Additionally, the Distractor Memory module relies on retrieving existing personas from a fixed dataset. The authors note that a promising future direction would be to dynamically generate optimal distractor personas on-the-fly during the conversation.

Summary updated

Abstract

We explore the task of improving persona consistency of dialogue agents. Recent models tackling consistency often train with additional Natural Language Inference (NLI) labels or attach trained extra modules to the generative agent for maintaining consistency. However, such additional labels and training can be demanding. Also, we find even the best-performing persona-based agents are insensitive to contradictory words. Inspired by social cognition and pragmatics, we endow existing dialogue agents with public self-consciousness on the fly through an imaginary listener. Our approach, based on the Rational Speech Acts framework (Frank and Goodman, 2012), can enforce dialogue agents to refrain from uttering contradiction. We further extend the framework by learning the distractor selection, which has been usually done manually or randomly. Results on Dialogue NLI (Welleck et al., 2019) and PersonaChat (Zhang et al., 2018) dataset show that our approach reduces contradiction and improves consistency of existing dialogue models. Moreover, we show that it can be generalized to improve context-consistency beyond persona in dialogues.
BibTeX
@inproceedings{kim2020will,
  author = {Hyunwoo Kim and Byeongchang Kim and Gunhee Kim},
  title = {{Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.65},
}
Figure