Introduction
Collaborative search is a common practice where multiple individuals coordinate to solve complex information tasks. Despite the prevalence of dedicated tools, most users prefer to conduct these activities within instant messaging (IM) platforms like Slack or Microsoft Teams. However, search queries in group chats are often "lightweight," fragmented, and heavily dependent on conversational context. For example, a user might request "more info on that," referring to a topic discussed much earlier in the thread.
Historically, bridging this gap required a "Wizard of Oz" approach, where human operators manually interpreted context and provided results. CoSearchAgent addresses this by leveraging the reasoning capabilities of Large Language Models (LLMs) to automate the entire collaborative search pipeline within existing IM environments.
System Architecture
CoSearchAgent is implemented as a Slack plugin using the Bolt-Python framework and GPT-3.5-turbo. The system operates through a three-stage pipeline designed to handle the nuances of multi-party dialogue.
- Query Processing: Upon being mentioned, the agent analyzes the preceding 20 messages to rewrite the user's request into a standalone, search-engine-optimized query. If ambiguity persists, the agent proactively asks a clarifying question.
- Search Result Presentation: Utilizing the Google Search API, the agent retrieves top results. Rather than displaying generic snippets, it crawls the HTML content (up to 5000 tokens) and uses the LLM to extract and summarize only the information relevant to the specific group query.
- Retrieval-Augmented Generation (RAG): The final stage synthesizes a comprehensive answer based on the extracted snippets. This response includes inline citations (e.g.,
[1],[2]) linking to source materials, which mitigates hallucinations and builds user trust.
Data Collection and Logging
While CoSearchAgent is a functional system rather than a pre-trained model, it serves as a powerful tool for generating research datasets. The system implements a comprehensive logging framework that captures multi-party dialogue history, generated search queries, retrieved results, and user click logs for future analysis.
Discussion and Key Insights
The development of CoSearchAgent highlighted several critical factors for successful collaborative AI assistants. First, contextual rewriting is essential; ignoring the chat history leads to search failure in group settings. By expanding queries based on history, the agent achieves semantic consistency comparable to human researchers.
Second, snippet replacement significantly improves the user experience. By replacing SEO-optimized snippets with LLM-generated summaries, users can evaluate relevance without leaving the chat interface. Finally, the agent's mixed-initiative interaction—asking for clarification rather than guessing—reduces the trial-and-error loop typical of complex searches.
Evaluation and Results
The authors evaluated CoSearchAgent by comparing it against three alternative modes: Direct Search, a human-led "Wizard of Oz" setup, and a version with Query Processing but no RAG.
The case study revealed that CoSearchAgent matches the accuracy of human "wizards" in query rewriting. Qualitatively, the RAG-based answers with citations were found to be superior to raw search results, providing direct answers while maintaining verifiability. The system demonstrates that LLMs can effectively manage the "heavy lifting" of multi-user interactions that previously necessitated human intervention.
Limitations and Future Work
Current limitations include the LLM's context window, which may restrict performance in exceptionally long research sessions. Additionally, the agent is currently limited to a single round of clarification to avoid intrusiveness. Future research will focus on large-scale user studies to quantify time savings and optimize the balance between agent proactivity and user experience.
Summary updated