Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations

Peng Qi, Yuhao Zhang, Christopher D. Manning

TL;DR

This paper tackles the challenge of generating informative and specific questions in information-seeking conversations where the questioner lacks access to the answer's context. They introduce novel metrics for informativeness and specificity and use reinforcement learning to train a model that asks better questions, significantly outperforming baselines in both automatic and human evaluations.

What Problem Does This Paper Address?

Conversations are a primary way humans exchange information. For AI systems to engage in natural, helpful conversations, they need to be able to ask good questions. Existing question generation models often assume knowledge of the answer or rely on predefined schemas, making them unsuitable for open-domain information-seeking scenarios where the questioner genuinely needs to acquire new, unknown information. This paper addresses the problem of generating questions that are not only informative (i.e., elicit new knowledge) but also specific to the ongoing conversation, avoiding generic or irrelevant queries. This is crucial for building AI systems that can truly reason about and interact with humans in a pragmatic way.

Methods

The authors propose a two-pronged approach:

  1. Question Generation Model: They build a sequence-to-sequence model that encodes the shared topic and conversation history to generate the next question. This model is initially trained using negative log-likelihood (NLL) to mimic observed questions.
  2. Informativeness and Specificity Metrics:
    • Informativeness: They define informativeness based on how much new information an answer provides, measured by its overlap with previously revealed answers. A question's informativeness is then the informativeness of the answer it elicits. This is operationalized using a pre-trained conversational QA model.
  3. Specificity: They train a classifier to distinguish true follow-up questions from generic or irrelevant ones. The score from this classifier serves as a measure of specificity.
  4. Reinforcement Learning for Optimization: The question generation model is then finetuned using reinforcement learning. A reward function is created by combining the informativeness and specificity scores. This reward signal guides the model to generate questions that are both informative and specific, moving beyond simply mimicking existing data. They use a self-critical sequence training approach and combine the RL objective with NLL to maintain fluency.

Datasets

The primary dataset used is QuAC (Question Answering in Context). The authors note that while other conversational QA datasets exist, QuAC is suitable because it aligns with their information-asymmetric setting where the questioner doesn't have access to the answer context. They repurpose the original development set as their test set due to the held-out nature of the official test set.

Key Insights and Ideas

  • Pragmatic Reasoning is Key: The core idea is that effective question generation in information-seeking dialogues requires pragmatic reasoning – understanding what information is needed and how to best acquire it, rather than just predicting the next likely question.
  • Informativeness Beyond Overlap: Defining informativeness not just by n-gram overlap with previous questions but by the novelty of the answer it elicits is a crucial distinction.
  • Specificity as a Counterbalance: While informativeness is important, it can lead to overly generic questions. The specificity metric acts as a vital counterbalance, ensuring questions are relevant and focused.
  • Reinforcement Learning for Goal-Oriented Generation: Using RL to optimize for these novel metrics allows the model to learn a more goal-directed behavior, actively seeking information rather than passively mimicking.
  • Human Evaluation Aligns with Metrics: The fact that human evaluations largely align with their proposed informativeness and specificity metrics provides strong validation for their approach.

Key Results

  • Improved Metrics: The finetuned model significantly outperforms the baseline on both the proposed informativeness (INFO) and specificity (SPEC) metrics, as well as conventional metrics like ROUGE-L. For example, on the test set, the model achieves an INFO score of 0.752 compared to the baseline's 0.664, and a SPEC score of 0.835 compared to the baseline's 0.779.
  • Human Preference: Human evaluators favored the authors' system over the baseline for informativeness (93.75% of the time) and overall quality (81.5%), and specificity (82%).
  • Reduced Repetition: Analysis shows the system reduces n-gram repetition in questions compared to the conversation history, indicating more novel and specific queries.

Limitations and Future Work

  • Generic Questions: The informativeness metric can still be exploited by generic questions (e.g., "What else do you know?"). However, the specificity reward helps mitigate this.
  • Unanswerable Questions: The current informativeness metric can be misled by questions that are unanswerable by the QA model, as seen in some reference questions in the QuAC dataset.
  • Sampling-based Decoding: The authors suggest that using sampling-based decoding strategies instead of greedy decoding could lead to more diverse and less repetitive generated questions, potentially improving human evaluation scores further.
  • Practical Pragmatics: The paper presents a "shallow implementation of pragmatics," suggesting deeper, more nuanced pragmatic reasoning remains an open area.

Summary updated

BibTeX
@inproceedings{qi2020stay,
  author = {Peng Qi and Yuhao Zhang and Christopher D. Manning},
  title = {{Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.3},
}
Figure