Abstract
Conversational Information Seeking (CIS) presents unique challenges due to the underspecified nature of user queries and the heavy reliance on dialogue context. This paper explores refinements to the multi-stage retrieval pipeline, introducing Paraphrase Query Expansion (PQE) to enhance initial document retrieval and Multi-view Point-wise Ranking (monoT5M) to improve re-ranking robustness. Our results on TREC CAsT benchmarks demonstrate that these innovations effectively mitigate the noise-context trade-off and data scarcity issues, leading to significant gains in retrieval precision and recall.
Introduction
Unlike traditional keyword-based web search, conversational search involves a series of interconnected utterances. This context-dependency often leads to queries that are ambiguous when viewed in isolation. For instance, a user might ask "How deadly is it?" referring to a disease mentioned several turns prior.
As illustrated in Figure 1, failing to correctly resolve context can lead to irrelevant retrieval results. Current multi-stage pipelines attempt to solve this by rewriting queries, but they often face a "noise-context trade-off" where adding history introduces distracting information. Furthermore, the scarcity of high-quality conversational training data often leads to models that overfit and perform poorly on imperfect automatic rewrites.
Methodology
The proposed system enhances the standard retrieval pipeline through three key components:
Paraphrase Query Expansion (PQE)
To improve the recall of the first-stage retriever, we introduce PQE. Using a T5 model fine-tuned on the Quora Question Pairs dataset, the system generates a paraphrase of the rewritten query. By combining the original rewrite with its paraphrase, the retriever captures a broader range of semantic variations and synonyms without the risk of introducing irrelevant terms from the conversation history.
Document Expansion
We employ the doc2query technique, where a model predicts potential questions a document might answer. These predicted questions are appended to the document index, helping to bridge the vocabulary gap between user queries and formal document text.
Multi-view Point-wise Ranking (monoT5M)
For final re-ranking, we move beyond the standard monoT5 model to a multi-view variant, monoT5M. This model is trained on two simultaneous tasks: relevance ranking (predicting if a document is relevant to a query) and query generation (translating a document back into the query). This dual-task approach serves as a regularizer, making the model more robust to the "noisy" or messy inputs typical of automatic query reformulations.
Experimental Results
The system was evaluated using the TREC Conversational Assistance Track (CAsT) 2020 and 2021 benchmarks.
As shown in Figure 6, the monoT5M model consistently outperforms the baseline monoT5 across various metrics. Notably, in the "Automatic Rewrite" scenario—which represents the most realistic and challenging setting—monoT5M shows significant improvements in nDCG@3, indicating higher precision in the top-ranked results. Furthermore, the combination of PQE with Part-of-Speech (POS) filtering, which focuses expansion on meaningful nouns and verbs, provides additional performance boosts.
On the CAsT 2021 dataset, the integration of PQE and POS-filtering improved the baseline BM25 retriever's Mean Average Precision (d-MAP) from 0.2363 to 0.2541 and Recall from 0.7487 to 0.7723.
Conclusion
This study demonstrates that targeted query expansion via paraphrasing and robust multi-view re-ranking can significantly improve conversational passage retrieval. By focusing on the user's immediate intent through PQE and regularizing the ranker with query generation, we can better handle the inherent noise of multi-turn dialogues. Future research will explore deeper integration of multi-turn dialogue properties to further bridge the gap between raw utterances and user information needs.
Summary updated