Abstract
Retrieval-Augmented Generation (RAG) systems often fail when user queries fall outside the scope of their internal knowledge base, leading to hallucinations triggered by irrelevant retrieved documents. This paper introduces a statistical framework using Goodness-of-Fit (GoF) tests to characterize query-knowledge relevance. By mapping high-dimensional embeddings to a one-dimensional test statistic, the proposed method identifies out-of-knowledge (OoK) queries in real-time and detects distribution shifts in user queries over time.
Introduction
The reliability of RAG systems depends on the relevance of retrieved context. When a system encounters an "out-of-knowledge" (OoK) query—one that cannot be answered by the provided corpus—it typically retrieves the "least irrelevant" documents. These distractors often cause Large Language Models (LLMs) to generate confident but false answers. Current systems lack efficient mechanisms to flag these queries online or to detect when the underlying knowledge base has become obsolete (offline).
Methodology
The authors propose treating query relevance as a statistical hypothesis testing problem. The core approach involves mapping text embeddings into a one-dimensional test statistic that measures how well a query fits the "In-Knowledge" (IK) distribution.
Online and Offline Testing Scenarios
The framework addresses two primary scenarios:
- Online Testing: A Goodness-of-Fit (GoF) test is used to evaluate individual queries in real-time. A query is rejected as OoK if its test statistic is a statistical outlier relative to a reference IK distribution.
- Offline Testing: To detect long-term distribution shifts, the two-sample Kolmogorov-Smirnov (KS) test compares a batch of recent queries against historical data.
To handle cases where a set of true IK queries is unavailable, the authors utilize LLMs to generate synthetic in-knowledge queries from corpus text chunks. These synthetic queries serve as a proxy for the system's knowledge boundaries, making the approach "training-free."
Experimental Setup
The framework was evaluated using eight QA datasets and two specialized medical corpora: a collection of 18 medical Textbooks and the PubMed abstract database. Ground truth benchmarks included MedQA-US and PubMedQA.
Key Results and Insights
The study yielded several critical insights into the behavior of RAG systems and the effectiveness of statistical relevance testing.
Detection Performance
The GoF test, particularly when using an Energy-based score, demonstrated high accuracy in distinguishing IK from OoK queries, often achieving an AUROC between 0.95 and 0.99. The approach is also highly sample-efficient, maintaining performance even with limited reference data.
LLMs as Relevance Judges
A striking finding was that GPT-3.5 and GPT-4 performed poorly at identifying OoK queries. These models often assigned high relevance scores to completely irrelevant context, highlighting the need for external statistical validation.
Distributional Separation
As shown in the frequency distributions, synthetic queries effectively mirror the distribution of real IK queries. This allows for clear thresholding to reject irrelevant inputs, as OoK queries (e.g., from TruthfulQA) form a distinct, non-overlapping distribution.
Limitations and Future Work
While effective, the current framework has limitations. Synthetic queries generated from single document chunks may be simpler than real-world queries requiring multi-document reasoning. Future work will explore "context-aware" chunking and the integration of faithfulness metrics to ensure that LLMs use retrieved information correctly.
Conclusion
This statistical framework provides a robust, training-free method for enhancing RAG reliability. By accurately identifying the boundaries of a system's knowledge, it prevents hallucinations at the source and provides a clear signal for when a knowledge base requires maintenance.
Summary updated