In a standard Retrieval-Augmented Generation (RAG) pipeline, a user provides a query, a system retrieves relevant documents, and a model generates an answer. This works well for isolated questions like "What is the capital of France?" However, human conversation is rarely isolated. We use pronouns ("What is its population?"), omit previously mentioned subjects (ellipsis), and shift topics mid-stream.
Traditional retrieval engines are "context-blind." They treat each query as an independent event. To bridge this gap, researchers have developed Contextual Query Reformulation (CQR)—a mechanism that transforms messy, interdependent conversational turns into standalone, "de-contextualized" search queries.
The Context Gap
The fundamental problem in conversational search is the "query gap." As noted in Query Rewriting in Retrieval-Augmented Large Language Models by Xinbei Ma and Nan Duan, there is a significant mismatch between the way humans ask follow-up questions and the way document retrievers index information.
If a user asks "How much does it cost?" after a long discussion about the iPhone 15, a retriever searching for "How much does it cost?" will likely return generic financial advice rather than product pricing. CQR solves this by "unpacking" the intent. As Ahmed Elgohary and Denis Peskov demonstrate with the CANARD dataset, the goal is to rewrite the question into a self-contained format: "How much does the iPhone 15 cost?"
Mechanisms of Reformulation
Modern systems typically insert a dedicated "Rewriter" module before the retrieval stage. This module, often a fine-tuned language model like T5 or a prompted LLM, performs several linguistic operations:
- Coreference Resolution: Replacing pronouns (it, they, she) with the specific entities they refer to. Xiaochen Zuo and Zhicheng Dou used this in their TREC 2019 entry to ensure keyword-based engines could find relevant documents.
- Ellipsis Completion: Filling in missing verbs or subjects that are implied by the previous turn.
- Topic Propagation: Identifying which parts of the history are still relevant. Ida Mele and Franco Maria Nardini showed in Topic Propagation in Conversational Search that detecting "topic shifts" is crucial to prevent the rewriter from injecting stale, irrelevant context into new questions.
Training the Rewriter
How do we teach a model to rewrite queries effectively? While early methods relied on human-annotated rewrites, recent work focuses on aligning the rewriter with the retriever's performance.
In IterCQR, Kang-il Lee and Kyomin Jung propose an iterative training framework. Instead of trying to match a "gold" human rewrite, the model is rewarded based on whether the rewritten query actually helps the retriever find the right answer. This often results in queries that are more effective for machines than those written by humans, as they may include specific keywords or structural markers that trigger better retrieval hits.
Scaling to Complex Domains
The utility of CQR extends beyond general web search into specialized fields:
- Business Intelligence: In ChatBI, Jinqing Lian uses a classification model to determine if a query is a follow-up, merging context only when necessary to reduce the computational cost of processing long histories.
- Technical Support: Yunyao Li describes how Summit Concierge uses chain-of-thought reasoning to resolve temporal ambiguities (e.g., "What's next?") by mapping them to absolute data points in a conference schedule.
- Collaborative Search: Peiyuan Gong and Jiaxin Mao demonstrate in CoSearchAgent that LLMs can even synthesize queries from multi-party chat histories, where the context is spread across multiple different speakers.
Why It Works
CQR works because it decouples dialogue understanding from information retrieval. By transforming a conversational turn into a standalone statement, we allow the retrieval component to remain "dumb" and efficient. The retriever doesn't need to understand the nuances of human linguistics; it only needs to match a well-formed, explicit query against its index. This modularity, as highlighted in the R3AG workshop, is what makes refined and reliable RAG possible in the messy reality of human interaction.