Abstract
In conversation, we often ask one-word questions such as 'Why?' or 'Who?'. Such questions are typically easy for humans to answer, but can be hard for computers, because their resolution requires retrieving both the right semantic frames and the right arguments from context. This paper introduces the novel ellipsis resolution task of resolving such one-word questions, referred to as sluices in linguistics. We present a crowd-sourced dataset containing annotations of sluices from over 4,000 dialogues collected from conversational QA datasets, as well as a series of strong baseline architectures.
1. Introduction
Human dialogue is characterized by efficiency, often achieved through ellipsis—the omission of words that are readily inferable from context. A prevalent form of ellipsis is sluicing, where a speaker poses a standalone wh- question (e.g., "When?" or "Where?"). While humans resolve these effortlessly, they present a significant challenge for natural language processing systems. Traditional approaches have treated sluice resolution as a span-prediction task, attempting to identify a contiguous text segment in the preceding turn. However, real-world dialogue often requires synthesizing information from disjointed parts of the context or performing grammatical transformations (e.g., shifting person from "I" to "you").
2. Methodology
The authors propose framing sluice resolution as a Natural Language Generation (NLG) task rather than simple span extraction. By treating resolution as a sequence-to-sequence (Seq2Seq) problem, models can generate full, grammatically correct questions that represent the speaker's intent.
Several architectures were evaluated:
- Copy & Edit Heuristics: Rule-based baselines that manipulate the previous question or answer.
- LSTM-Seq2Seq: A standard RNN-based encoder-decoder model.
- Transformer: A standard multi-headed self-attention model.
- GPT-2: A pre-trained language model (117M parameters), evaluated both zero-shot and fine-tuned on the task-specific dataset.
The input format consists of the previous question ($Q_1$), the previous answer ($A_1$), and the target sluice ($Q_2$), with the model tasked to generate the expanded resolution ($R$).
3. The Sluice Resolution Dataset
To support this task, the authors developed a novel dataset by extracting one-word follow-up questions from the QuAC and CoQA conversational QA datasets. The resulting 4,175 datapoints were annotated via Amazon Mechanical Turk, where workers provided full-sentence resolutions for each sluice. Notably, the dataset accounts for the inherent ambiguity of dialogue by allowing for multiple valid resolutions.
4. Results and Analysis
Fine-tuned GPT-2 (FT) emerged as the superior model across all metrics. It achieved a BLEU score of 0.391 and a GLEU of 0.348, outperforming both rule-based and standard Seq2Seq baselines. In human evaluations using Mean Reciprocal Rank (MRR), GPT-2 (FT) scored 0.529. Remarkably, in 19% of cases ($r_1 = 0.190$), human judges preferred the model's generated resolution over the human-annotated "gold" standard, often due to the model's ability to generate more expressive or contextually informative phrasing.
4.1. Attention Dynamics
Analysis of the Transformer's attention mechanisms revealed a strategic "Wh- Attention Shift." When generating the initial word of a resolution, the model primarily attends to the sluice itself at the end of the input sequence. As generation continues, attention shifts back to the preceding context to extract the necessary semantic frames and arguments.
5. Conclusion and Future Work
This work demonstrates that generative models are highly effective at resolving elliptical sluices in conversation, surpassing traditional span-prediction methods. While the models perform well within a three-turn context window, challenges remain for long-distance dependencies. Future research may expand the scope to more complex elliptical forms and explore the impact of resolution on downstream conversational agents, potentially reducing the frequency of uninformative "I don't know" responses.
Summary updated