K2Q: Generating Natural Language Questions from Keywords with User Refinements

Zhicheng Zheng, Xiance Si, Edward Y. Chang, Xiaoyan ZhuIJCNLP 2011 · OpenReview

Abstract: Zhicheng Zheng, Xiance Si, Edward Chang, Xiaoyan Zhu. Proceedings of 5th International Joint Conference on Natural Language Processing.

2011.

Introduction

Keyword search is the standard for modern information retrieval, yet it often fails in Community Question Answering (CQA) contexts like Quora or Yahoo! Answers. Keywords are inherently ambiguous; for instance, a search for "New York restaurant" could indicate a desire for dining recommendations or an inquiry into local tipping etiquette. CQA systems require specific, well-articulated natural language questions to provide relevant answers.

The K2Q (Keywords to Questions) system bridges this gap by using a novel "User Inquiry Intent" (UII) model. It generates candidate questions and suggests refinement words to help users clarify their intent, addressing three core challenges: generating "unseen" questions (98% of questions in the dataset are unique), ranking candidates without frequency data, and providing effective refinements based on information gain rather than simple frequency.

Methodology: The User Inquiry Intent (UII) Model

The core of K2Q is the UII model, a probabilistic graphical model that treats user intent as a latent variable generating both messy keyword queries and structured natural language questions.

The User Inquiry Intent (UII) model, a probabilistic graphical model describing the generation of keywords and questions from a latent intent.

Key technical components include:

  • Template-Based Generation: To handle novel questions, K2Q induces templates by replacing specific words with clusters (grouped via k-means). This allows the system to generate patterns like "What is the meaning of [WordCluster]?"
  • Adaptive Language Modeling: A modified Kneser-Ney smoothing approach calculates slot transition probabilities, managing the sparsity of natural language data.
  • Entropy-Based Refinement: The system suggests words that maximize "information gain" (entropy reduction) to narrow down user intent efficiently, ensuring diverse suggestions rather than redundant frequent terms.
  • Ranking: Questions are ranked by the joint probability of the question and keywords under the UII model.

Experimental Evaluation

The model was trained and evaluated using a massive dataset: 12.8 million questions from CQA sites, WikiAnswers for testing generalization, and a one-week snapshot of Google query logs (500,000 query-question pairs) as ground truth.

The UII model demonstrated significant improvements over standard N-gram language models (LM). As shown in the table below, the UII model achieved a 43.5% recall at top-1, representing a 16% relative improvement over the baseline.

Comparison of ranking accuracy (Recall at top k) between the baseline Language Model (LM) and the proposed UII model.

Further results indicated that using 1-variable templates improved candidate question recall by 6.6% to 16.2%, confirming the system's ability to synthesize relevant questions not present in the training set. Additionally, the interactive refinement process reached target questions more effectively than frequency-based methods within five iterations.

Conclusion

The K2Q system successfully leverages latent intent to transform ambiguous keywords into precise natural language questions. While current parameter estimation is performed separately for different components, future work aims for global joint optimization and real-time human-in-the-loop trials to further refine the refinement suggestions and incorporate search log data more directly into training.

Summary updated

BibTeX
@inproceedings{zheng2011k,
  author = {Zhicheng Zheng and Xiance Si and Edward Y. Chang and Xiaoyan Zhu},
  title = {{K2Q: Generating Natural Language Questions from Keywords with User Refinements}},
  booktitle = {IJCNLP},
  year = {2011},
  url = {https://openreview.net/forum?id=c3MH8yh1y6f},
}
Figure