TL;DR
State-of-the-art dialogue agents frequently contradict their assigned personas because standard decoding objectives prioritize local language fluency over long-term consistency. This paper introduces a plug-and-play framework that endows agents with "public self-consciousness" via an imaginary listener, drastically reducing contradictions on-the-fly without requiring expensive Natural Language Inference (NLI) labels or extra training modules.
What Problem Does This Paper Address?
Endowing open-domain dialogue agents with a persistent "persona" (e.g., "I am a construction worker who loves cats") helps them generate more engaging responses. However, even advanced generative models like Blender or GPT-based architectures remain highly insensitive to contradictions. An agent might say "I like going outside" and follow it up immediately with "I like to stay at home."
Existing methods fix this by training separate Natural Language Inference (NLI) models to classify pairs of utterances as entailment, neutral, or contradiction, using these scores to rerank outputs. However, this supervised approach is data-hungry, computationally expensive, and generalizes poorly due to dataset-specific distribution shifts. This paper addresses the challenge of maintaining persona consistency without relying on NLI labels or modifying the underlying generative model's parameters.
Methods
The authors draw inspiration from social cognition: humans maintain consistency by predicting how they are perceived by others—a trait known as public self-consciousness. To mimic this, they adapt the Rational Speech Acts (RSA) framework, treating token generation as a recursive game between an agent and an imaginary listener.
The system operates token-by-token during decoding:
- Base Speaker ($S_0$): A standard pre-trained conditional dialogue model provides a baseline probability distribution for the next token.
- Imaginary Listener ($L_0$): Using Bayes' rule, the listener evaluates how likely a candidate token is to help them correctly guess the speaker's true persona out of a small set of choices (the true persona plus "distractor" personas).
- Self-Conscious Speaker ($S_1$): The final token distribution combines the base speaker's fluency with the listener's feedback. Tokens that make the correct persona obvious receive a probability bonus.
Learnable Distractor Selection
A key novelty is how the framework selects the alternative "distractor" personas that the listener uses to test the speaker. Rather than choosing them randomly, the authors implement a Distractor Memory (DM) module using a life-long memory network. It uses a fine-tuned BERT query network to retrieve personas from the training set that are contextually relevant but distinct, forcing the agent to generate highly specific, non-contradictory language.
Datasets
The approach was evaluated using three primary datasets:
- Dialogue NLI: An evaluation set of 542 dialogue instances explicitly annotated with entailing, neutral, and contradictory utterance candidates to test ranking performance.
- PersonaChat: A benchmark dataset containing 17,878 training and 1,000 validation conversations conditioned on explicit persona profiles.
- EmpatheticDialogues: Used to evaluate the model's ability to generalize to context-consistency (staying consistent with dialogue history rather than a fixed persona profile).
Key Insights and Ideas
- Language Models are Blind to Persona Context: Perplexity analysis revealed that standard models exhibit significantly lower perplexity for contradictory words than for the correct persona words. They behave like plain language models, prioritizing local syntactic patterns over global persona constraints.
- Dissimilar Distractors Work Best: When training the Distractor Memory module, selecting distractors that were semantically farthest from the target persona yielded better consistency than selecting nearest neighbors. Contrast helps the imaginary listener better adjudicate the agent's identity.
- Dynamic Prior Updates Matter: Instead of using a static uniform prior over personas at every step, updating the listener's world prior dynamically based on previously generated tokens preserves cumulative context and prevents the model from drifting into contradictions mid-sentence.
Key Results
The self-conscious framework ($S_1$ + DM) was evaluated across three strong base models: ControlSeq2Seq, TransferTransfo, and Blender.
- Contradiction Reduction: On the Dialogue NLI dataset, the proposed method cut contradiction rates (
Contradict@1) by more than half for all models. For instance, Blender's contradiction rate dropped from 42.4% to 19.6%. - Accuracy Boost: The accuracy of selecting the ground-truth utterance (
Hits@1) improved significantly across the board (e.g., from 11.1% to 18.8% for TransferTransfo). - Outperforming NLI Baselines: Remarkably, the unsupervised $S_1$ + DM agent outperformed standard base models that used fully supervised, explicitly trained NLI rerankers on the
Hits@1metric. - Human Evaluation: Human judges on Amazon Mechanical Turk rated the self-conscious agent as significantly more consistent than the baseline models while maintaining an equivalent level of conversational engagingness.
Limitations and Future Work
While highly effective, the framework currently relies on manual grid searches to tune the speaker and listener rationality coefficients ($\alpha$ and $\beta$). If these values are set too high, the language model degenerates, simply copying fragments of the persona text verbatim rather than generating natural dialogue.
Additionally, the Distractor Memory module relies on retrieving existing personas from a fixed dataset. The authors note that a promising future direction would be to dynamically generate optimal distractor personas on-the-fly during the conversation.
Summary updated