SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation

Junkai Zhou, Liang Pang, Huawei Shen, Xueqi Cheng

Abstract

Persona-based chatbots often fail to maintain a consistent identity or provide coherent responses despite the fluency of underlying Large Language Models (LLMs). This paper introduces SimOAP (Simple Over-sampling and Post-evaluation), a model-agnostic strategy that improves dialogue quality without expensive retraining. By generating a large pool of candidate responses and employing a two-step filtering process—TF-IDF for coherence and Natural Language Inference (NLI) for persona consistency—SimOAP effectively identifies high-quality responses that standard decoding methods often overlook.

Introduction

In open-domain dialogue, maintaining a specific persona (e.g., "I am a vegan librarian who loves jazz") is a significant challenge. LLMs frequently suffer from inconsistency, where they contradict their assigned traits, and incoherence, where responses are irrelevant to the conversation history.

Existing solutions often involve retraining models with complex loss functions, which is computationally expensive. A key observation driving this work is the "Probability Paradox": high-quality, consistent responses are often present within a model's output distribution but are rarely assigned the highest probability. Standard decoding methods like greedy search or top-p sampling typically prioritize word-level likelihood, which does not always correlate with high-level persona alignment.

The Probability Paradox: The ratio of good responses increases as the rank of the selected response in the probability distribution increases, suggesting that high-quality responses are often hidden at lower probability ranks.

Methodology: The SimOAP Framework

SimOAP is a "plug-in" strategy consisting of two primary stages: Over-sampling and Post-evaluation.

1. Over-sampling Stage

Instead of generating a few candidates, SimOAP generates a large batch (e.g., 2,000) using top-k sampling. To address the computational cost of such large-scale generation, the authors utilize Model Acceleration. This involves using smaller "proxy" models (e.g., DistilGPT2 or BERT-medium) to generate the initial candidate pool rapidly. This shift moves the focus from teaching the model to be better to better searching the model's existing capabilities.

2. Post-evaluation Stage

The candidate pool is refined through a coarse-to-fine filtering process:

  • Coherence Evaluation (Fast): A fast TF-IDF similarity check compares candidates against the dialogue history to eliminate irrelevant responses.
  • Consistency Evaluation (Precise): The top 100 coherent candidates are processed by a Natural Language Inference (NLI) model (RoBERTa fine-tuned on DialogueNLI). This model evaluates whether a response entails the assigned persona, selecting the candidate with the highest alignment score.
The SimOAP architecture: Over-sampling using a proxy model followed by a two-step evaluation for coherence and consistency.

Experimental Results

The strategy was evaluated on the PersonaChat dataset using backbone models like BoB (BERT-over-BERT) and Multi-GPT2.

  • Automatic Metrics: SimOAP significantly improved consistency scores. For the BoB model, the consistency score (C) rose from 0.114 to 0.579. Perplexity (PPL) also decreased, indicating improved fluency.
  • Human Evaluation: Blind human tests confirmed these findings, with SimOAP-generated responses receiving higher ratings for fluency, coherence, informativeness, and consistency. Specifically, the consistency rating for the BoB model rose from 0.20 to 0.70.
  • Efficiency: By using proxy models, the accelerated version of SimOAP achieved a 3x speedup over traditional reranking methods like MMI (Maximum Mutual Information) while maintaining superior performance.

Conclusion and Future Work

SimOAP demonstrates that improving persona-based dialogue is not just about better training, but about better searching of a model's existing capabilities. While the method introduces some latency due to the multi-step evaluation, model acceleration makes it a viable strategy for many applications. Future work will focus on further reducing latency and extending the method to long-form text generation.

Summary updated

Abstract

Language models trained on large-scale corpora can generate remarkably fluent results in open-domain dialogue. However, for the persona-based dialogue generation task, consistency and coherence are also key factors, which are great challenges for language models. Existing works mainly focus on valuable data filtering, model structure modifying, or objective function designing, while their improvements are limited and hard to generalize to all types of pre-trained language models. However, we find that language models can produce consistent and coherent responses if we consider enough generations. Thus, the problems lay in large-scale response generation and target response selection. In this work, a simple but effective two-stage SimOAP strategy is proposed, i.e., over-sampling and post-evaluation. The over-sampling stage takes large-scale responses from existing trained models efficiently via off-the-shelf distilling and compressing methods, and the post-evaluation stage selects a good response based on multiple well-designed evaluation metrics from large-scale candidates. Experimental results show that the proposed plug-in SimOAP strategy improves the backbone models and outperforms the baseline strategies in both automatic and human evaluations.
BibTeX
@inproceedings{zhou2023simoap,
  author = {Junkai Zhou and Liang Pang and Huawei Shen and Xueqi Cheng},
  title = {{SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.553},
}
Figure