Abstract
Modern conversational assistants excel at simple, factoid-based queries but often struggle with the complex, multi-turn interactions required for exploratory information seeking. This research addresses the disconnect between user intent and system execution by proposing a framework that maps "speech acts" (the functional intent of a user's utterance) to "search actions" (the specific operations performed by a search engine). Using a multi-channel Attention-based Deep Neural Network (ADNN) and a newly developed dataset, CONVEX, this study demonstrates that combining transformer-based embeddings with linguistic features and conversational metadata significantly improves the prediction of system-level actions in conversational search.
Introduction
While voice-activated assistants like Siri and Alexa have become ubiquitous, their utility is largely confined to "factoid" queries (e.g., "What is the height of the Eiffel Tower?") or simple command executions. However, human information seeking is often non-factoid and exploratory, resembling the nuanced back-and-forth dialogue found in a reference library. In these scenarios, a single query is rarely sufficient. Instead, users engage in a discourse where their needs evolve through clarification and refinement.
This paper identifies a critical gap in current conversational systems: the lack of a formal connection between the natural language used by humans and the technical actions a search system must perform. To bridge this gap, the researchers propose a model that identifies the functional aspects of discourse—speech acts—and uses them to trigger appropriate system-level search actions, such as query creation or document scanning.
Methodology
Data Collection: The Wizard-of-Oz Study
To capture authentic human-system interaction, the researchers conducted a Wizard-of-Oz (WoZ) study involving 25 users across 75 sessions. Participants interacted with what they believed was an AI, but was actually a human "Wizard"—an experienced searcher following strict protocols. This approach allowed for the collection of realistic dialogue data in a controlled environment.
Thematic Analysis and Annotation
The resulting dialogues were manually annotated using the Conversational Roles (COR) model. The researchers identified:
- 12 Speech Acts: Including categories such as Question/Seek, Clarify, and Repeat.
- 4 Search Actions: Including Query Creation, SERP (Search Engine Results Page) Scanning, and Document Scanning.
The ADNN Model
The core of the proposed solution is an Attention-based Deep Neural Network (ADNN). This model utilizes a multi-channel architecture to process three distinct types of input simultaneously:
- Word-level features: Leveraging DistilBERT for efficient natural language understanding.
- Linguistic features: Utilizing the SpaCy API to extract parts-of-speech and syntactic dependencies.
- Metadata: Incorporating contextual information such as the utterance number, the speaker's role, and the previous speech act.
Experimental Setup and Datasets
The study utilized two primary datasets for training and validation:
- CONVEX (CONversation with EXplanations): A new "gold standard" dataset created by the authors, containing 1,834 utterances and 509 system actions derived from the WoZ study.
- SCS (Spoken Conversational Search): An existing, publicly available dataset used to test the generalizability of the models across different conversational contexts.
Key Results and Discussion
Classification Performance
The multi-channel model demonstrated strong performance in identifying user intent but faced greater challenges in predicting system actions:
- Speech Act Classification: The integrated model (Meta + Linguistic + BERT) achieved a maximum accuracy of 90.2% on the CONVEX dataset and 72.7% on the SCS dataset.
- Search Action Classification: Accuracy was notably lower, peaking at 58.8% on CONVEX and 61.1% on SCS.
The Role of Context
A significant finding of the study was the importance of metadata. Ablation studies revealed that removing conversational history (the previous speech act) caused the most substantial drop in performance for search action prediction. This suggests that while the current utterance tells the system what is being said, the conversational context tells the system why it is being said and what should happen next.
Multi-Channel Synergy
The research highlights that while transformer models like BERT are powerful, they are not a panacea for specialized domains. The best results consistently came from combining BERT with "hand-engineered" linguistic and contextual features. This synergy is essential for understanding the nuances of search-related discourse.
Limitations and Future Directions
The primary hurdle identified by the authors is data scarcity; while CONVEX is a valuable contribution, deep learning models typically require larger datasets to achieve high accuracy in complex tasks like search action prediction. Additionally, class imbalance—where certain speech acts occur much more frequently than others—remains a challenge.
Future work will focus on data augmentation techniques, such as SMOTE, to balance the dataset. The ultimate goal is to integrate this pipeline into a live, proactive conversational agent capable of recognizing topic changes and offering intelligent query modifications in real-time digital library environments.
Conclusion
This research provides a foundational framework for moving beyond reactive conversational assistants. By formalizing the link between speech acts and search actions, the study paves the way for systems that can truly partner with users in complex information-seeking tasks, understanding not just the words spoken, but the functional intent behind them.
Summary updated