Abstract
Conversational Information Seeking (CIS) represents a paradigm shift in information retrieval, moving from static "query-response" interactions to dynamic, multi-turn dialogues. This paper synthesizes the current state of CIS, unifying search, recommendation, and question answering into a single interactive framework. We examine the core architectural components—including state tracking, dense retrieval, and mixed-initiative logic—while highlighting the critical role of Large Language Models (LLMs) and the ongoing challenges in evaluation, memory, and factual reliability.
1. Introduction
For decades, information retrieval (IR) has been dominated by the "query box" model: a user enters a keyword, and the system returns a list of links. However, human information seeking is naturally iterative and conversational. Users often begin with vague needs that require clarification or evolve as they acquire new information.
CIS addresses the challenge of building systems that can handle these complex, multi-turn interactions. This involves solving several fundamental sub-problems: maintaining context and memory across turns, determining when a system should take initiative (proactivity), adapting to different modalities (such as voice-only interfaces), and developing new metrics to evaluate the success of a non-linear conversation.
2. System Architecture and Methods
A high-level architecture for CIS systems typically consists of four core components:
- Conversational User Interfaces (CUIs): These range from simple text-based chatbots to multi-modal systems that incorporate sensor data and touch.
- State Tracking: Modern systems utilize Transformer-based models (such as BERT and T5) to perform Conversational Query Rewriting (CQR). This process converts context-dependent follow-up questions (e.g., "What about in Australia?") into standalone, unambiguous queries based on the dialogue history.
- Response Ranking and Generation: The field is moving away from traditional link lists toward Dense Retrieval (vector-based search) and Generative Summarization. Large Language Models (LLMs) are increasingly employed to generate fluent, natural language answers grounded in retrieved documents.
- Mixed-Initiative Logic: Systems often employ Reinforcement Learning (RL) to decide the optimal "action" at any given turn—whether to provide a direct answer, ask for clarification, or elicit further user preferences.
3. Benchmarks and Datasets
The evolution of CIS has been shaped by several essential benchmarks:
- TREC CAsT (2019-2022): The primary standard for evaluating conversational passage retrieval.
- QuAC & CoQA: Datasets focused on context-aware question answering where answers are extracted as text spans.
- Qulac & MIMICS: Specialized datasets for training systems to generate effective clarifying questions.
- Multi-WOZ: A popular dataset for task-oriented dialogues, such as booking travel or services.
- TopiOCQA: A newer benchmark designed to test a system's ability to handle topic switching within a single conversation.
4. Key Insights
The Convergence of IR Tasks
In a conversational setting, search, recommendation, and QA are no longer distinct silos. A system helping a user find a product is simultaneously searching a corpus, recommending items based on inferred style, and answering specific questions about product features.
The "Local Context" Trap
Most current models rely heavily on the last 2–3 turns of a conversation. Achieving human-level understanding requires "long-term" memory and the ability to reference information from much earlier in the session, a capability that current architectures still struggle to maintain.
Listenability vs. Readability
In voice-only CIS, the serial nature of audio means systems cannot simply read a list of results. Responses must be adapted for "listenability," adjusting length and prosody (tone/pitch) to avoid overwhelming the user’s cognitive load.
The Risks of Mixed-Initiative
While proactivity is a major goal, an over-eager system that asks too many questions can frustrate users and erode trust. Identifying the "opportune moment" to take initiative remains a significant research frontier.
5. Performance and Limitations
- The NLU Gap: Despite the power of state-of-the-art Transformers, there remains a 20-40% performance gap between automatic query rewriting and manual human rewrites in benchmarks like TREC CAsT.
- Utility of Clarification: Research indicates that asking just two well-targeted clarifying questions can improve recommendation accuracy by up to 25% in closed-domain scenarios.
- The Impact of LLMs: While models like ChatGPT have revolutionized conversational fluency, "hallucination" (the generation of false information) remains a critical barrier for information-seeking tasks where accuracy is paramount.
6. Future Directions
To move the field forward, several bottlenecks must be addressed. Current offline evaluation methods often suffer from "false negatives," failing to reward valid answers not present in the ground-truth dataset. Additionally, the requirement for deep interaction memory raises significant privacy concerns. Future work must focus on personalized evaluation that accounts for individual user expertise and the development of high-precision CIS for specialized domains like healthcare and enterprise search.
Summary updated