Abstract
Retrieval-Augmented Large Language Models (RALMs) have made significant strides in enhancing the accuracy of generated responses. However, existing research often overlooks the data quality issues within retrieval results, often caused by inaccurate existing vector-distance-based retrieval methods. We propose to boost the precision of RALMs' answers from a data quality perspective through the Context-Driven Index Trimming (CDIT) framework, where Context Matching Dependencies (CMDs) are employed as logical data quality rules to capture and regulate the consistency between retrieved contexts. Based on the semantic comprehension capabilities of Large Language Models (LLMs), CDIT can effectively identify and discard retrieval results that are inconsistent with the query context and further modify indexes in the database, thereby improving answer quality. Experiments demonstrate performance on challenging question-answering tasks. Also, the flexibility of CDIT is verified through its compatibility with various language models and indexing methods, which offers a promising approach to bolster RALMs' data quality and retrieval precision jointly.
Introduction
The "Retrieval" in Retrieval-Augmented Generation (RAG) is the foundation of the system's accuracy. Most modern RALMs rely on vector similarity—converting text into high-dimensional embeddings and assuming that proximity in vector space equates to semantic similarity. However, this approach suffers from a critical flaw: semantic inversion. Because embedding models are often trained on word co-occurrence, sentences with opposite factual meanings (e.g., "The patient survived" vs. "The patient died") often end up as close neighbors in vector space. When a retriever pulls these conflicting contexts into an LLM's prompt, it introduces noise that leads to hallucinations or incorrect answers.
This paper introduces Context-Driven Index Trimming (CDIT), a framework that treats retrieval as a data quality problem. Rather than simply re-ranking results, CDIT identifies logical contradictions and permanently "trims" faulty links within the vector database index.
The CDIT Framework
CDIT operates through three primary mechanisms designed to bridge the gap between coarse vector distance and fine-grained symbolic logic.
1. Context Matching Dependencies (CMDs)
Drawing inspiration from classical database theory—specifically Matching Dependencies—the authors define Context Matching Dependencies (CMDs). These are logical rules used to regulate consistency. For instance, a CMD might dictate that if two text segments share the same subject and object, their predicates must be semantically consistent for them to be considered valid matches for a specific query.
2. LLM-as-a-Logic-Engine
To enforce these CMDs, CDIT utilizes a "Determiner" LLM (such as GPT-3.5). The model decomposes retrieved sentences into linguistic components: Subject, Predicate, and Object. It then evaluates these components against the CMD rules to determine if the retrieved context actually supports the query or contradicts it. This allows the system to use the semantic reasoning of the LLM to "audit" the mathematical output of the vector retriever.
3. The Witness Theorem and Index Trimming
The most innovative aspect of CDIT is its ability to permanently improve the underlying data structure. If a query $q$ is found to be consistent with document $A$ but inconsistent with document $B$, the query serves as a "witness" to a faulty link in the vector database. Once a sufficient number of witnesses are identified, CDIT performs "Index Trimming." It removes the similarity edge between $A$ and $B$ in graph-based structures like Hierarchical Navigable Small World (HNSW) graphs. This ensures that future queries will not be led down the same path of conflicting information, making the database cleaner over time.
Experimental Evaluation
The authors evaluated CDIT across four challenging factual and reasoning benchmarks using the April 2018 English Wikipedia dump as the knowledge base:
- ARC-Challenge: Multiple-choice reasoning requiring deep domain knowledge.
- PubHealth: Fact-checking for public health claims.
- PopQA: Entity-relationship questions (subject, relation, object).
- TriviaQA-unfiltered: Complex, compositional questions requiring high retrieval precision.
Key Results and Insights
- Performance Gains: CDIT improved the accuracy of various LLMs (including Llama 2/3, Mistral, and Falcon) by an average of 3.75%. The most significant leap was a 15.21% improvement for the Falcon-7b model on the TriviaQA dataset.
- Index Flexibility: The framework proved robust across different indexing strategies, including
IndexFlatL2,IndexHNSWFlat, andIndexIVFFlat, demonstrating its utility as a general-purpose enhancement for vector databases. - Synergy with Advanced RAG: When integrated with frameworks like Self-RAG, CDIT provided an additional 3.62% average improvement, suggesting it is a complementary "plug-and-play" module rather than a replacement for existing RAG optimizations.
- Permanent Data Quality: Unlike re-ranking, which is a transient per-query operation, index trimming modifies the database itself. This creates a system that effectively "learns" from its retrieval errors.
Limitations and Future Directions
While CDIT offers a powerful new perspective, it faces challenges with long-form text where Subject-Predicate-Object extraction becomes complex. Additionally, the current reliance on manually crafted CMD rules suggests a need for future research into "CMD Mining"—the automatic discovery of logical rules from raw data. Finally, while the LLM-based trimming process introduces a one-time computational cost, it significantly streamlines the quality of future retrievals.
Conclusion
CDIT represents a shift from model-centric to data-centric RAG. By treating the vector index as a dynamic, correctable structure and applying logical dependencies, CDIT successfully mitigates the pitfalls of semantic inversion and enhances the precision of Retrieval-Augmented Large Language Models.
Summary updated