Query Understanding via Intent Description Generation

Ruqing Zhang, Jiafeng Guo, Yixing Fan, Yanyan Lan, Xueqi Cheng

Introduction

In Information Retrieval (IR), "query understanding" is a fundamental challenge. When a user enters a search term like "Amazon rain forest," their specific intent—whether it concerns location, age, or conservation efforts—remains ambiguous. Traditionally, AI systems address this through coarse-grained methods like query classification or clustering. However, these methods often fail to capture the nuance of a user's specific information need.

Researchers have introduced a new task called Query-to-Intent-Description (Q2ID). This task moves beyond simple categorization to generate full, natural language descriptions of search intent. By framing this as a "reverse ranking" task, the system uses a set of documents (both relevant and irrelevant) to "triangulate" and describe exactly what a user is seeking.

Methodology: Contrastive Generation (CtrsGen)

The proposed solution, CtrsGen (Contrastive Generation), is built on a Sequence-to-Sequence (Seq2Seq) framework specifically tailored for the complexities of search data. Unlike standard summarization models that only process relevant information, CtrsGen leverages both "hits" and "misses" to define the boundaries of a query's intent.

The CtrsGen architecture features a hierarchical encoding structure and a specialized contrast-attention mechanism to distinguish relevant signals from noise.

Key Innovations

  • Hierarchical Encoding: The model processes multiple documents by encoding words into sentences and sentences into a comprehensive "mega-document" representation.
  • Query-Aware Encoder Attention: This mechanism uses the original search query to highlight sentences within relevant documents that are most likely to reveal the core topic.
  • Contrast-Based Decoder Attention: This is the model's standout feature. During generation, the model calculates a contrast score, prioritizing information present in relevant documents but absent or different in irrelevant ones. This allows the model to ignore "noisy" topics that might appear in both sets, such as overlapping keywords that do not define the specific intent.

Dataset and Experimental Setup

To evaluate the Q2ID task, a new benchmark dataset was constructed by repurposing public IR collections from TREC (2004 Robust and 2015–2017 Dynamic Domain Tracks) and SemEval (2015 and 2016 Task 3). The final dataset contains 5,358 queries, each paired with a human-written intent description and an average of 10.8 relevant and 65.5 irrelevant documents.

Key Results

CtrsGen was compared against several baselines, including extractive models like LexRank and state-of-the-art abstractive summarizers like Pointer-Generator (PG) networks.

Performance comparison of CtrsGen against various extractive and abstractive baselines, showing superior performance across all ROUGE metrics.

As shown in the results, CtrsGen achieved a ROUGE-L score of 20.21, significantly outperforming the best baseline, SD2 (18.49). Human evaluation further supported these findings; in terms of "Reasonableness" (capturing the true intent), CtrsGen scored 2.86/5, a substantial improvement over the best baseline's 1.73/5.

Ablation studies confirmed that the inclusion of irrelevant documents is vital. Removing the contrastive component led to a significant performance drop, proving that knowing what is not relevant is just as important for defining specific intent as knowing what is.

Conclusion and Future Work

The CtrsGen model demonstrates that treating intent generation as the inverse of document ranking provides a powerful way to "explain" search results. While the current model uses binary relevance (relevant vs. irrelevant), future work aims to incorporate graded relevance to better reflect the spectrum of search results. This technology envisions a future where search engines provide "intent snippets," helping users refine their searches more effectively by explicitly stating the focus of the provided results.

Summary updated

BibTeX
@inproceedings{zhang2020query,
  author = {Ruqing Zhang and Jiafeng Guo and Yixing Fan and Yanyan Lan and Xueqi Cheng},
  title = {{Query Understanding via Intent Description Generation}},
  booktitle = {ACM International Conference on Information and Knowledge Management},
  year = {2020},
  url = {https://openreview.net/forum?id=FdWh44Qk5t},
}
Figure