In a natural conversation, humans are remarkably efficient. We use pronouns like "it" or "him" (anaphora) and omit entire subjects when the context is clear (ellipsis). If you ask a friend about "The Bronze Age collapse" and follow up with "What were the causes?", your friend knows exactly what "the causes" refers to.
However, as Yu Shi, Chenyan Xiong, and colleagues point out in Few-Shot Generative Conversational Query Rewriting, standard search engines are "optimized for standalone, keyword-rich queries." When a user provides a fragmented conversational turn, the retrieval system lacks the necessary signal to find relevant documents. The research direction of Conversational Query Rewriting (CQR) aims to bridge this gap by transforming "messy," context-dependent dialogue turns into self-contained, "decontextualized" search queries.
The Core Challenge: Anaphora and Ellipsis
The primary bottleneck in conversational search is not the retrieval algorithm itself, but the underspecified nature of the input. As noted in ConvRAG by L.X. Ye and others, resolving linguistic dependencies is often more critical for performance than the specific search algorithm used.
Without a mechanism to resolve these dependencies, a system sees a sequence of turns as isolated events. Early attempts to solve this, such as the Historical Query Expansion (HQE) method proposed by Jheng-Hong Yang and Sheng-Chieh Lin in Query and Answer Expansion from Conversation History, used training-free heuristics to inject terms from previous turns into the current query. While effective, these methods often introduced noise by including irrelevant terms from the dialogue history.
From Heuristics to Generative Reformulation
The field shifted toward using generative models to "rewrite" the query. The intuition is to treat the problem as a translation task: translating a "conversational" language into "search engine" language.
Sheng-Chieh Lin and Jimmy Lin demonstrated in Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Models that models like T5 can effectively resolve coreferences and ellipses. By feeding the model the current turn plus the dialogue history, the model generates a standalone version of the question (e.g., "What were the causes of the Bronze Age collapse?").
However, a key realization emerged: what is clear to a human is not always what is best for a search engine.
Aligning with Retriever Preferences
Traditional rewriting models were trained to mimic human-written "gold" rewrites. But human rewrites are designed for readability, not necessarily for maximizing retrieval scores.
This led to the development of frameworks like CONQRR, where Zeqiu Wu and Hannah Rashkin used reinforcement learning to reward the rewriter based on actual retrieval performance. Similarly, AdaQR by Tianhua Zhang and James R. Glass uses the probability of the correct answer as a reward signal. These "retriever-aware" models learn to include specific keywords or entities that help the search engine, even if the resulting query looks slightly unnatural to a human.
Scaling and Efficiency with LLMs
As Large Language Models (LLMs) became more capable, researchers began using them to handle increasingly complex conversational contexts. CHIQ, developed by Fengran Mo and colleagues, uses a two-step process to "clean up" and enrich history before rewriting, allowing smaller open-source models to match the performance of GPT-4.
To address the high cost of calling LLMs during every search turn, researchers are now looking at distillation. DiSCo by Mohammad Aliannejadi and Evangelos Kanoulas introduces a "relaxed" distillation objective. Instead of trying to make a small model generate the exact same text as an LLM, it trains the model to match the similarity scores of LLM-rewritten queries. This allows the system to maintain high retrieval accuracy while remaining fast enough for real-time interaction.
Beyond Simple Rewriting: Clarification
Sometimes, even with the full history, a query remains ambiguous. In these cases, the system shouldn't just guess; it should ask for clarification. AGENT-CQ by Clemencia Siro and others uses an ensemble of LLMs to generate and evaluate clarifying questions. This ensures that the system only proceeds when it has a clear, decontextualized understanding of the user's information need, effectively turning the search process itself into a collaborative dialogue.