Introduction
While Large Language Models (LLMs) have demonstrated remarkable capabilities across various natural language tasks, their tendency to "hallucinate"—generating factually incorrect but plausible-sounding information—remains a significant barrier to deployment in high-stakes environments. Retrieval-Augmented Generation (RAG) has emerged as a primary solution, grounding LLM outputs in external knowledge. However, the efficacy of RAG is fundamentally limited by the quality of the retrieval mechanism. Traditional systems often struggle with the "lexical chasm" (where users and documents use different terms) and the "semantic mismatch" (where meaning is captured but specific technical details are lost).
Methodology: The Hybrid Retrieval Module ($Ret_{Hyb-RRF}$)
To address these limitations, this research proposes a three-stage hybrid retrieval pipeline designed to optimize context relevance and mitigate hallucinations.
1. Query Expansion (QE)
The pipeline begins by expanding the user query ($q$) using WordNet to incorporate synonyms. This step is crucial for bridging the lexical gap between the user's phrasing and the terminology used in the target documents.
2. Dynamic Weighting
A key innovation of this approach is the "specificity switch." Rather than applying a fixed weight to retrieval methods, the system calculates a specificity score ($S(q')$) for the expanded query using TF-IDF.
- Specific Queries: For queries containing rare or technical terms, the system assigns a higher weight (0.7) to the Sparse Retriever (BM25) and a lower weight (0.3) to the Dense Retriever.
- Generic Queries: For conceptual or broad queries, the weights are reversed, prioritizing the semantic capabilities of the Dense Retriever (utilizing
all-mpnet-base-v2).
3. Weighted Reciprocal Rank Fusion (RRF)
The ranked results from both retrievers are fused using a weighted RRF algorithm. This ensures that documents ranked highly by both methods—or exceptionally high by the prioritized method—are surfaced to the LLM. The final context is then fed into a LLaMA-3-8B-Instruct model for generation.
Experimental Evaluation
The proposed system was evaluated using HaluBench, a comprehensive benchmark comprising 13,867 samples from diverse domains including finance (FinanceBench), medicine (PubMedQA, COVIDQA), and general reasoning (DROP, HaluEval). For the generation phase, performance was validated against a manually annotated subset of 300 high-quality instances to ensure rigorous assessment of the LLM's faithfulness.
Key Results
The hybrid retrieval approach demonstrated superior performance across all metrics compared to standalone sparse or dense methods:
- Retrieval Accuracy: The hybrid model achieved a Mean Average Precision (MAP@3) of 0.897, significantly higher than sparse (0.724) and dense (0.768) retrieval.
- Hallucination Mitigation: The impact was most pronounced in specialized domains. In medical queries (PubMed), the hybrid approach dropped the hallucination rate to 4%, compared to 16% for sparse and 20% for dense retrieval.
- Robustness on Complex Tasks: On samples where LLMs typically struggle (the "FAIL" labels in HaluBench), the hybrid pipeline reached an accuracy of 77.24%, nearly double that of the sparse-only approach (33.33%).
Conclusion
The integration of dynamic weighting and query expansion within a hybrid retrieval framework significantly enhances the faithfulness of LLM responses. By intelligently navigating the trade-off between lexical precision and semantic depth, the $Ret_{Hyb-RRF}$ module provides a robust "hallucination insurance" policy, particularly for technical domains like finance and medicine. Future research will explore the application of this framework to larger model architectures and the integration of advanced cross-encoder re-ranking steps.
Summary updated