QueryExplorer: An Interactive Query Generation Assistant for Search and Exploration

Kaustubh D. Dhole, Shivam Bajaj, Ramraj Chandradevan, Eugene Agichtein

Abstract

Formulating effective search queries remains a challenging task, particularly when users lack expertise in a specific domain or are not proficient in the language of the content. Providing example documents of interest might be easier for a user. However, such query-by-example scenarios are prone to concept drift, and the retrieval effectiveness is highly sensitive to the query generation method, without a clear way to incorporate user feedback. To enable exploration and to support Human-In-The-Loop experiments we propose QueryExplorer– an interactive query generation, reformulation, and retrieval interface with support for HuggingFace generation models and PyTerrier’s retrieval pipelines and datasets, and extensive logging of human feedback. To allow users to create and modify effective queries, our demo supports complementary approaches of using LLMs interactively, assisting the user with edits and feedback at multiple stages of the query formulation process. With support for recording fine-grained interactions and user annotations, QueryExplorer can serve as a valuable experimental and research platform for annotation, qualitative evaluation, and conducting Human-in-the-Loop (HITL) experiments for complex search tasks where users struggle to formulate queries.

Introduction

Crafting precise search queries is a non-trivial task even for domain experts. This difficulty is exacerbated when users explore unfamiliar domains—such as legal or medical archives—or search across language barriers. While "Query-by-Example" (QBE) offers a potential solution by allowing users to provide a relevant document as a seed, it often suffers from concept drift, where the retrieval system focuses on peripheral aspects of the document rather than the user's core intent. Furthermore, traditional Information Retrieval (IR) tools often function as "black boxes," making it difficult for researchers to analyze how users interact with and refine generated queries.

System Architecture

QueryExplorer is designed as a modular web interface, built using HuggingFace’s Gradio, that bridges the gap between LLM-powered generation and robust retrieval pipelines. The system architecture, as shown in Figure 1, integrates three primary functional components:

  1. Query Generation (QG): Leveraging models like flan-t5-xxl, the system performs zero-shot or few-shot query generation based on a user-provided example document.
  2. Query Reformulation (QR): The system suggests expansion terms and keywords to sharpen the query's focus and improve retrieval effectiveness.
  3. Retrieval Pipeline: Powered by PyTerrier, the backend supports standard algorithms like BM25 and allows for the integration of custom neural re-rankers.
System workflow of QueryExplorer illustrating the interaction between the searcher/annotator and the underlying LLM and retrieval components.

Interactive Workflow and User Interface

The core strength of QueryExplorer lies in its Human-In-The-Loop (HITL) design. Rather than being passive recipients of search results, users are active participants in the query formulation process. As illustrated in the user interface (Figure 2), the workflow follows a structured path:

  • Input: The user provides a text snippet or document of interest.
  • Generation: The system generates multiple candidate queries.
  • Refinement: The user can select, edit, or combine these queries.
  • Feedback: Upon viewing retrieved documents, users can provide explicit relevance feedback, which the system uses to further refine the search via LLM-driven reformulation.
The QueryExplorer User Interface, demonstrating the transition from an example document to generated queries and retrieved results.

Research and Experimental Capabilities

QueryExplorer is not just a search tool but a platform for IR research. It includes a dedicated Configuration Tab that allows researchers to swap LLM models, modify prompts, and adjust retrieval hyperparameters dynamically.

A critical feature for experimental use is Granular Interaction Logging. The system records every user action—including query edits, document dwell times, and relevance annotations. This data is invaluable for studying searcher behavior and evaluating the effectiveness of different HITL strategies. The system is data-agnostic, supporting standard benchmarks via ir_datasets and providing few-shot exemplars from datasets like MSMarco.

Conclusion and Future Work

QueryExplorer demonstrates an end-to-end pipeline for interactive, LLM-assisted search. By allowing users to "steer" the query generation process, it effectively mitigates the concept drift common in traditional QBE systems. While current limitations include the computational overhead of large LLMs and potential model biases, the tool provides a robust framework for future research. Future work will focus on using the collected interaction data to train more efficient, specialized query generators and conducting large-scale user studies to quantify the impact of HITL refinement on search precision and recall.

Summary updated

Abstract

Formulating effective search queries remains a challenging task, particularly when users lack expertise in a specific domain or are not proficient in the language of the content. Providing example documents of interest might be easier for a user. However, such query-by-example scenarios are prone to concept drift, and the retrieval effectiveness is highly sensitive to the query generation method, without a clear way to incorporate user feedback. To enable exploration and to support Human-In-The-Loop experiments we propose QueryExplorer– an interactive query generation, reformulation, and retrieval interface with support for Hug-gingFace generation models and PyTerrier’sretrieval pipelines and datasets, and extensivelogging of human feedback. To allow users to create and modify effective queries, our demo supports complementary approaches of using LLMs interactively, assisting the user with edits and feedback at multiple stages of the query formulation process. With support for recording fine-grained interactions and user annotations, QueryExplorer can serve as a valuable experimental and research platform for annotation, qualitative evaluation, and conducting Human-in-the-Loop (HITL) experiments for complex search tasks where users struggle to formulate queries.
BibTeX
@inproceedings{dhole2024queryexplorer,
  author = {Kaustubh D. Dhole and Shivam Bajaj and Ramraj Chandradevan and Eugene Agichtein},
  title = {{QueryExplorer: An Interactive Query Generation Assistant for Search and Exploration}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  url = {https://openreview.net/forum?id=JeryB2pqZR},
}
Figure