Abstract
HEISIR is a training-free framework designed to optimize Conversational Data Retrieval (CDR) by extracting structured semantic quadruplets (Subject-Verb-Object-Adjunct) during the data ingestion phase. By shifting the computational burden of Large Language Model (LLM) semantic analysis to offline indexing, HEISIR achieves state-of-the-art retrieval performance while maintaining the low latency required for real-time applications.
Introduction
Retrieving information from past conversations—Conversational Data Retrieval (CDR)—is significantly more complex than standard document retrieval. Dialogues are inherently messy, characterized by multiple intents within single utterances, heavy speaker-dependency, and a reliance on multi-turn context.
Current approaches generally fall into two categories: dense/sparse retrieval methods (e.g., DPR, BM25), which often require extensive labeled data and fine-tuning, and LLM-based methods (e.g., HyDE, Re-ranking), which offer high contextual understanding but suffer from prohibitive "query-time" latency. HEISIR addresses these issues by providing a training-free system that combines LLM intelligence with the speed of standard vector search by moving reasoning to the ingestion phase.
Methodology
HEISIR introduces a novel two-step Data Ingestion Phase that utilizes LLMs to pre-process conversations into structured indices:
- Hierarchical Triplets Formulation: Messages are decomposed into core Subject-Verb-Object (SVO) triplets. To resolve ambiguity, the "Subject" is fixed to the speaker of the utterance, and non-semantic noise like tenses and auxiliaries is removed.
- Adjunct Augmentation: These triplets are expanded into Subject-Verb-Object-Adjunct (SVOA) quadruplets. Adjuncts capture critical contextual details such as "why," "where," or "how" (e.g., "because of a bug"). This hierarchical approach ensures core meaning is captured before adding descriptive complexity, reducing potential index noise.
During the Retrieval Phase, HEISIR employs a composite scoring mechanism. Rather than a simple query-to-message comparison, it calculates similarity across the entire conversation, specific messages, and the extracted SV, SVO, and SVOA components. Since these components are pre-embedded during ingestion, the search remains a high-speed vector similarity operation.
Experimental Results
The framework was evaluated across five diverse dialogue datasets, including BAD, DICES, Daily Dialog, PILD, and USS. HEISIR consistently outperformed both standard baselines and fine-tuned models.
As shown in the results, HEISIR (using GPT-3.5-turbo and OpenAI-large embeddings) achieved an acc@1 of 0.4085, significantly surpassing the best fine-tuned baseline (0.3533). Furthermore, HEISIR maintains high efficiency, with retrieval latency ranging from 0.07s to 0.13s, compared to 2–3 seconds for other LLM-based methods like HyDE or LameR.
Discussion and Conclusion
HEISIR demonstrates that moving LLM reasoning to the ingestion phase allows for high-accuracy, low-latency conversational retrieval without the need for labeled training data. The resulting SVOA indices provide human-readable interpretability, allowing developers to see exactly which semantic component triggered a match. Beyond retrieval, the framework is versatile enough for downstream tasks like intent and topic analysis.
Future work will focus on adapting the framework for non-English syntax and optimizing component weights (e.g., SV vs. SVOA) to further enhance performance. While highly effective for dialogue, applying this structured approach to general documents remains a challenge due to the lack of clear speaker-as-subject structures.
Summary updated