Introduction
In the field of Conversational Information Seeking (CIS), the ability of a system to ask natural, context-aware questions is paramount. Traditional approaches often fall into two traps: they either rank pre-existing "self-contained" questions, which sound repetitive and robotic in a dialogue, or they generate new questions from scratch, which frequently results in ungrammatical or irrelevant output. This paper introduces RISE (Reinforcement Iterative Sequence Editing), a framework designed to bridge this gap by treating conversational question generation as an iterative editing task optimized for minimum Levenshtein distance.
The Challenge of Conversational Question Simplification (CQS)
Conversational Question Simplification (CQS) involves taking a "heavy," self-contained question and refining it into a natural conversational form. For instance, instead of asking, "Was anyone opposed to Ira Hayes revealing the truth?", a human would likely ask, "Was anyone opposed to him revealing this?".
The primary technical hurdle is that standard training methods like Maximum Likelihood Estimation (MLE) treat every word with equal weight. Since most words in a simplified question are identical to the original, MLE-trained models often become "lazy," failing to learn the sparse but critical conversational edits such as anaphora (pronoun usage) and ellipsis (omitting redundant information).
The RISE Framework
RISE reframes CQS as a Hierarchical Combinatorial Markov Decision Process (HCMDP). Rather than generating a sentence from a blank slate, the model learns to edit the input question through two distinct sub-policies:
- Editing MDP: An encoder-based policy that determines the action for each token: Keep (K), Delete (D), Insert (I), or Substitute (S).
- Phrasing MDP: A decoder-based policy that selects the specific words to use when the Editing MDP triggers an "Insert" or "Substitute" action.
To optimize this process, the authors developed Iterative Reinforce Training (IRT) and Dynamic Programming based Sampling (DPS). IRT allows the model to explore various edit sequences and receive rewards based on how closely the result matches the target (minimizing edit distance). DPS addresses the context-dependency of edits, using a dynamic programming matrix to track expected edits and improve sampling efficiency during training.
Experimental Setup and Results
The model was evaluated on two key benchmarks: CANARD, a large-scale dataset for conversational rewriting, and CAsT (TREC Conversational Assistance Track), used to test generalization on unseen data.
RISE achieved state-of-the-art (SOTA) performance across multiple metrics. On CANARD, it outperformed the strongest baseline, QuerySim, by approximately 4% in ROUGE-L. More importantly, RISE demonstrated exceptional robustness on the unseen CAsT data. While other neural models often see performance degradation when moving to new domains, RISE maintained high scores, outperforming QuerySim by 1.3% in BLEU-4. Ablation studies confirmed that both the DPS sampling method and the Minimum Levenshtein Distance (MLD) objective were critical to these gains.
Discussion and Future Work
By focusing on the "delta"—the specific changes required to make a question conversational—RISE preserves core semantics better than "from-scratch" generation. Its iterative nature allows for multi-pass refinement, such as replacing a name with a pronoun in one pass and removing redundant phrases in the next.
However, limitations remain. In roughly 5% of cases, the model suffers from a "stop problem," where it continues editing a perfect sentence until it loses meaning or grammatical integrity. Future work aims to incorporate a discriminator to help the model recognize when a question has reached its optimal conversational form without sacrificing information.
Conclusion
RISE represents a significant step forward in making conversational agents sound more human. By optimizing for edit distance rather than word-for-word prediction, the framework successfully captures the subtle linguistic shifts that define natural dialogue.
Summary updated