Abstract
Persona-based chatbots often fail to maintain a consistent identity or provide coherent responses despite the fluency of underlying Large Language Models (LLMs). This paper introduces SimOAP (Simple Over-sampling and Post-evaluation), a model-agnostic strategy that improves dialogue quality without expensive retraining. By generating a large pool of candidate responses and employing a two-step filtering process—TF-IDF for coherence and Natural Language Inference (NLI) for persona consistency—SimOAP effectively identifies high-quality responses that standard decoding methods often overlook.
Introduction
In open-domain dialogue, maintaining a specific persona (e.g., "I am a vegan librarian who loves jazz") is a significant challenge. LLMs frequently suffer from inconsistency, where they contradict their assigned traits, and incoherence, where responses are irrelevant to the conversation history.
Existing solutions often involve retraining models with complex loss functions, which is computationally expensive. A key observation driving this work is the "Probability Paradox": high-quality, consistent responses are often present within a model's output distribution but are rarely assigned the highest probability. Standard decoding methods like greedy search or top-p sampling typically prioritize word-level likelihood, which does not always correlate with high-level persona alignment.
Methodology: The SimOAP Framework
SimOAP is a "plug-in" strategy consisting of two primary stages: Over-sampling and Post-evaluation.
1. Over-sampling Stage
Instead of generating a few candidates, SimOAP generates a large batch (e.g., 2,000) using top-k sampling. To address the computational cost of such large-scale generation, the authors utilize Model Acceleration. This involves using smaller "proxy" models (e.g., DistilGPT2 or BERT-medium) to generate the initial candidate pool rapidly. This shift moves the focus from teaching the model to be better to better searching the model's existing capabilities.
2. Post-evaluation Stage
The candidate pool is refined through a coarse-to-fine filtering process:
- Coherence Evaluation (Fast): A fast TF-IDF similarity check compares candidates against the dialogue history to eliminate irrelevant responses.
- Consistency Evaluation (Precise): The top 100 coherent candidates are processed by a Natural Language Inference (NLI) model (RoBERTa fine-tuned on DialogueNLI). This model evaluates whether a response entails the assigned persona, selecting the candidate with the highest alignment score.
Experimental Results
The strategy was evaluated on the PersonaChat dataset using backbone models like BoB (BERT-over-BERT) and Multi-GPT2.
- Automatic Metrics: SimOAP significantly improved consistency scores. For the BoB model, the consistency score (C) rose from 0.114 to 0.579. Perplexity (PPL) also decreased, indicating improved fluency.
- Human Evaluation: Blind human tests confirmed these findings, with SimOAP-generated responses receiving higher ratings for fluency, coherence, informativeness, and consistency. Specifically, the consistency rating for the BoB model rose from 0.20 to 0.70.
- Efficiency: By using proxy models, the accelerated version of SimOAP achieved a 3x speedup over traditional reranking methods like MMI (Maximum Mutual Information) while maintaining superior performance.
Conclusion and Future Work
SimOAP demonstrates that improving persona-based dialogue is not just about better training, but about better searching of a model's existing capabilities. While the method introduces some latency due to the multi-step evaluation, model acceleration makes it a viable strategy for many applications. Future work will focus on further reducing latency and extending the method to long-form text generation.
Summary updated