Learning to Ask Conversational Questions by Optimizing Levenshtein Distance

Zhongkun Liu, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Maarten de Rijke, Ming Zhou

Introduction

In conversational information seeking, systems must interact with users in a way that feels natural and context-aware. A common challenge is Conversational Question Simplification (CQS)—the task of transforming standalone, self-contained questions into natural dialogue turns. This typically involves introducing linguistic features such as anaphora (using pronouns like "he" or "it") and ellipsis (omitting redundant words). For instance, a system might simplify "Was anyone opposed to Ira Hayes revealing the truth?" into "Was anyone opposed to him in this?"

The Problem with Maximum Likelihood Estimation

Existing sequence-to-sequence models for CQS primarily rely on Maximum Likelihood Estimation (MLE). However, because the input and target questions in CQS are often 90% identical, MLE tends to treat all tokens equally. This causes the model to become "lazy," learning to copy the input perfectly while failing to master the sparse, critical edits—like swapping a specific name for a pronoun—that actually make the question conversational.

Comparison of CQS tasks and the conceptual difference between MLE and MLD objectives.

The RISE Framework

To overcome the limitations of MLE, the authors propose optimizing the Minimum Levenshtein Distance (MLD). MLD focuses explicitly on the number of insertions, deletions, or substitutions required to transform one text into another. Since MLD is discrete and non-differentiable, the authors introduce Reinforcement Iterative Sequence Editing (RISE), a framework that treats CQS as a Hierarchical Combinatorial Markov Decision Process.

RISE utilizes a BERT2BERT architecture governed by two policies:

  1. Editing Policy: Predicts token-level actions such as Keep (K), Delete (D), Insert (I), or Substitute (S).
  2. Phrasing Policy: Generates the specific replacement phrases when an 'I' or 'S' action is selected.

To train RISE effectively without step-by-step ground truth, the authors developed Iterative Reinforce Training (IRT). They also introduced Dynamic Programming based Sampling (DPS) to handle the interdependency of edits, allowing for much smarter exploration during reinforcement learning than standard methods.

Experimental Results

The model was evaluated on the CANARD dataset (for in-domain performance) and the CAsT dataset (for zero-shot generalization). RISE achieved state-of-the-art results across both, outperforming strong baselines like QuerySim and Trans++.

Performance comparison of RISE against state-of-the-art baselines on CANARD and CAsT datasets.

On the CANARD dataset, RISE reached 71.6% BLEU-4 and 86.2% ROUGE-L. Its superior generalization was evident on the unseen CAsT dataset, where it maintained a lead with 66.8% BLEU-4. Ablation studies highlighted that the MLD objective and DPS strategy are both essential; removing MLD caused a 3-7% drop in BLEU-4, while removing DPS led to a catastrophic failure in the RL training process.

Conclusion

By shifting the optimization focus from token-level likelihood to edit-level distance, RISE effectively addresses the "lazy" copying problem in conversational rewriting. The framework's ability to perform iterative, sparse edits allows it to generate more natural questions that generalize well to new domains. Future work aims to incorporate adversarial discriminators to further ensure that semantic meaning is preserved during the simplification process.

Summary updated

BibTeX
@inproceedings{liu2021learning,
  author = {Zhongkun Liu and Pengjie Ren and Zhumin Chen and Zhaochun Ren and Maarten de Rijke and Ming Zhou},
  title = {{Learning to Ask Conversational Questions by Optimizing Levenshtein Distance}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.438},
}
Figure