An Exploration Study of Multi-stage Conversational Passage Retrieval: Paraphrase Query Expansion and Multi-view Point-wise Ranking

Jia-Huei Ju, Chih-Ting Yeh, Cheng-Wei Lin, Chia-Ying Tsao, Jun-En Ding, Chuan-Ju Wang, Ming-Feng TsaiTREC 2021 · OpenReview

Abstract

Conversational Information Seeking (CIS) presents unique challenges due to the underspecified nature of user queries and the heavy reliance on dialogue context. This paper explores refinements to the multi-stage retrieval pipeline, introducing Paraphrase Query Expansion (PQE) to enhance initial document retrieval and Multi-view Point-wise Ranking (monoT5M) to improve re-ranking robustness. Our results on TREC CAsT benchmarks demonstrate that these innovations effectively mitigate the noise-context trade-off and data scarcity issues, leading to significant gains in retrieval precision and recall.

Introduction

Unlike traditional keyword-based web search, conversational search involves a series of interconnected utterances. This context-dependency often leads to queries that are ambiguous when viewed in isolation. For instance, a user might ask "How deadly is it?" referring to a disease mentioned several turns prior.

Figure 1: Example of conversational turns demonstrating the challenge of maintaining context and the potential for irrelevant system responses (r3).

As illustrated in Figure 1, failing to correctly resolve context can lead to irrelevant retrieval results. Current multi-stage pipelines attempt to solve this by rewriting queries, but they often face a "noise-context trade-off" where adding history introduces distracting information. Furthermore, the scarcity of high-quality conversational training data often leads to models that overfit and perform poorly on imperfect automatic rewrites.

Methodology

The proposed system enhances the standard retrieval pipeline through three key components:

Paraphrase Query Expansion (PQE)

To improve the recall of the first-stage retriever, we introduce PQE. Using a T5 model fine-tuned on the Quora Question Pairs dataset, the system generates a paraphrase of the rewritten query. By combining the original rewrite with its paraphrase, the retriever captures a broader range of semantic variations and synonyms without the risk of introducing irrelevant terms from the conversation history.

Document Expansion

We employ the doc2query technique, where a model predicts potential questions a document might answer. These predicted questions are appended to the document index, helping to bridge the vocabulary gap between user queries and formal document text.

Multi-view Point-wise Ranking (monoT5M)

For final re-ranking, we move beyond the standard monoT5 model to a multi-view variant, monoT5M. This model is trained on two simultaneous tasks: relevance ranking (predicting if a document is relevant to a query) and query generation (translating a document back into the query). This dual-task approach serves as a regularizer, making the model more robust to the "noisy" or messy inputs typical of automatic query reformulations.

Experimental Results

The system was evaluated using the TREC Conversational Assistance Track (CAsT) 2020 and 2021 benchmarks.

Figure 6: Performance comparison (nDCG@3 and nDCG@1000) for monoT5 vs monoT5M across Manual and Automatic query rewrite settings, including the impact of PQE and POS filtering.

As shown in Figure 6, the monoT5M model consistently outperforms the baseline monoT5 across various metrics. Notably, in the "Automatic Rewrite" scenario—which represents the most realistic and challenging setting—monoT5M shows significant improvements in nDCG@3, indicating higher precision in the top-ranked results. Furthermore, the combination of PQE with Part-of-Speech (POS) filtering, which focuses expansion on meaningful nouns and verbs, provides additional performance boosts.

On the CAsT 2021 dataset, the integration of PQE and POS-filtering improved the baseline BM25 retriever's Mean Average Precision (d-MAP) from 0.2363 to 0.2541 and Recall from 0.7487 to 0.7723.

Conclusion

This study demonstrates that targeted query expansion via paraphrasing and robust multi-view re-ranking can significantly improve conversational passage retrieval. By focusing on the user's immediate intent through PQE and regularizing the ranker with query generation, we can better handle the inherent noise of multi-turn dialogues. Future research will explore deeper integration of multi-turn dialogue properties to further bridge the gap between raw utterances and user information needs.

Summary updated

BibTeX
@inproceedings{ju2021exploration,
  author = {Jia-Huei Ju and Chih-Ting Yeh and Cheng-Wei Lin and Chia-Ying Tsao and Jun-En Ding and Chuan-Ju Wang and Ming-Feng Tsai},
  title = {{An Exploration Study of Multi-stage Conversational Passage Retrieval: Paraphrase Query Expansion and Multi-view Point-wise Ranking}},
  booktitle = {TREC},
  year = {2021},
  url = {https://openreview.net/forum?id=LXwucrdAT3},
}
Figure