Learning to Ask Conversational Questions by Optimizing Levenshtein Distance

Zhongkun Liu, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Maarten de Rijke, Ming Zhou

Introduction

In the field of Conversational Information Seeking (CIS), the ability of a system to ask natural, context-aware questions is paramount. Traditional approaches often fall into two traps: they either rank pre-existing "self-contained" questions, which sound repetitive and robotic in a dialogue, or they generate new questions from scratch, which frequently results in ungrammatical or irrelevant output. This paper introduces RISE (Reinforcement Iterative Sequence Editing), a framework designed to bridge this gap by treating conversational question generation as an iterative editing task optimized for minimum Levenshtein distance.

The Challenge of Conversational Question Simplification (CQS)

Conversational Question Simplification (CQS) involves taking a "heavy," self-contained question and refining it into a natural conversational form. For instance, instead of asking, "Was anyone opposed to Ira Hayes revealing the truth?", a human would likely ask, "Was anyone opposed to him revealing this?".

The process of Conversational Question Simplification (CQS) involves transforming a self-contained question into a natural conversational one through edits like anaphora and ellipsis.

The primary technical hurdle is that standard training methods like Maximum Likelihood Estimation (MLE) treat every word with equal weight. Since most words in a simplified question are identical to the original, MLE-trained models often become "lazy," failing to learn the sparse but critical conversational edits such as anaphora (pronoun usage) and ellipsis (omitting redundant information).

The RISE Framework

RISE reframes CQS as a Hierarchical Combinatorial Markov Decision Process (HCMDP). Rather than generating a sentence from a blank slate, the model learns to edit the input question through two distinct sub-policies:

  1. Editing MDP: An encoder-based policy that determines the action for each token: Keep (K), Delete (D), Insert (I), or Substitute (S).
  2. Phrasing MDP: A decoder-based policy that selects the specific words to use when the Editing MDP triggers an "Insert" or "Substitute" action.

To optimize this process, the authors developed Iterative Reinforce Training (IRT) and Dynamic Programming based Sampling (DPS). IRT allows the model to explore various edit sequences and receive rewards based on how closely the result matches the target (minimizing edit distance). DPS addresses the context-dependency of edits, using a dynamic programming matrix to track expected edits and improve sampling efficiency during training.

Experimental Setup and Results

The model was evaluated on two key benchmarks: CANARD, a large-scale dataset for conversational rewriting, and CAsT (TREC Conversational Assistance Track), used to test generalization on unseen data.

Performance comparison of RISE against baselines on CANARD and CAsT datasets, showing state-of-the-art results and superior generalization.

RISE achieved state-of-the-art (SOTA) performance across multiple metrics. On CANARD, it outperformed the strongest baseline, QuerySim, by approximately 4% in ROUGE-L. More importantly, RISE demonstrated exceptional robustness on the unseen CAsT data. While other neural models often see performance degradation when moving to new domains, RISE maintained high scores, outperforming QuerySim by 1.3% in BLEU-4. Ablation studies confirmed that both the DPS sampling method and the Minimum Levenshtein Distance (MLD) objective were critical to these gains.

Discussion and Future Work

By focusing on the "delta"—the specific changes required to make a question conversational—RISE preserves core semantics better than "from-scratch" generation. Its iterative nature allows for multi-pass refinement, such as replacing a name with a pronoun in one pass and removing redundant phrases in the next.

However, limitations remain. In roughly 5% of cases, the model suffers from a "stop problem," where it continues editing a perfect sentence until it loses meaning or grammatical integrity. Future work aims to incorporate a discriminator to help the model recognize when a question has reached its optimal conversational form without sacrificing information.

Conclusion

RISE represents a significant step forward in making conversational agents sound more human. By optimizing for edit distance rather than word-for-word prediction, the framework successfully captures the subtle linguistic shifts that define natural dialogue.

Summary updated

Abstract

Conversational Question Simplification (CQS) aims to simplify self-contained questions into conversational ones by incorporating some conversational characteristics, e.g., anaphora and ellipsis. Existing maximum likelihood estimation (MLE) based methods often get trapped in easily learned tokens as all tokens are treated equally during training. In this work, we introduce a Reinforcement Iterative Sequence Editing (RISE) framework that optimizes the minimum Levenshtein distance (MLD) through explicit editing actions. RISE is able to pay attention to tokens that are related to conversational characteristics. To train RISE, we devise an Iterative Reinforce Training (IRT) algorithm with a Dynamic Programming based Sampling (DPS) process to improve exploration. Experimental results on two benchmark datasets show that RISE significantly outperforms state-of-the-art methods and generalizes well on unseen data.
BibTeX
@inproceedings{liu2021learning,
  author = {Zhongkun Liu and Pengjie Ren and Zhumin Chen and Zhaochun Ren and Maarten de Rijke and Ming Zhou},
  title = {{Learning to Ask Conversational Questions by Optimizing Levenshtein Distance}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.438},
}
Figure