InteGround: On the Evaluation of Verification and Retrieval Planning in Integrative Grounding

Jay J. Cheng, Qianqian Zhuang, Haoran Li, Chunkit Chan, Xin Liu, Lin Qiu, Yangqiu Song2025 · DOI

Abstract

Integrative grounding is the challenge of retrieving and verifying multiple interdependent pieces of evidence to support complex queries. We present InteGround, an evaluation framework designed to systematically measure model performance in these tasks. Our analysis reveals that while Large Language Models (LLMs) handle redundant information well, they often "rationalize" by using internal knowledge to fill gaps in incomplete evidence. Furthermore, we show that logical planning through premise abduction, especially when paired with iterative self-reflection, significantly outperforms standard retrieval methods.

Introduction

Most current Retrieval-Augmented Generation (RAG) systems are evaluated using simple queries that can be answered by a single document. However, real-world information needs are frequently complex, requiring the synthesis of multiple, linked facts. This paper addresses the gap in evaluating "integrative grounding"—the ability of a system to find all necessary evidence and recognize when provided evidence is insufficient.

Conceptual overview of integrative grounding vs. naive retrieval.

Without robust evaluation, LLMs remain prone to hallucinations when encountering "sub-optimal" evidence conditions, such as missing facts or high noise levels.

The InteGround Framework

The InteGround framework focuses on two primary components: Groundedness Verification and Retrieval Planning.

Groundedness Verification

This task tests whether a model can accurately determine if a set of evidence supports a hypothesis across four distinct scenarios:

  • Informative: All necessary ground-truth evidence is present.
  • Redundant: Ground-truth evidence is mixed with distracting facts.
  • Incomplete: Only a partial subset of the necessary evidence is provided.
  • Uninformative: Only irrelevant distractors are provided.

Retrieval Planning

We evaluate several strategies for reformulating queries to locate interdependent evidence:

  • Query Expansion: Using methods like HyDE to generate hypothetical answers.
  • Decomposition: Breaking claims into atomic or propositional verifiable facts.
  • Premise Abduction: Generating the logical premises required to prove a claim and searching for them.
  • Self-Reflection: An iterative step where the model identifies missing information before subsequent searches.

Datasets

We repurposed 1,625 items from four diverse domains: EntailmentBank (scientific reasoning), WiCE (Wikipedia claim verification), HotpotQA (multi-hop QA), and MuSiQue (complex multi-step questions).

Key Findings

The Rationalization Trap

A critical finding is that LLMs tend to "rationalize" when faced with incomplete evidence. Instead of identifying the missing information, they often incorrectly label a claim as supported by drawing on their internal training data. This behavior poses a significant reliability risk for RAG applications. Supervised Natural Language Inference (NLI) models were found to be more "honest" verifiers than LLMs in these scenarios.

Logic-Driven Retrieval and Reflection

Our experiments demonstrate that undirected planning can degrade performance by introducing noise. In contrast, premise abduction emerged as the most effective planning strategy. By focusing on the logical requirements of a claim, it creates a more directed search.

Furthermore, the addition of a zero-shot self-reflection step—asking "What am I still missing?"—consistently improved retrieval quality across all tested strategies and datasets. For example, on the MuSiQue dataset using a BM25 retriever, reflection turned failing planning strategies into ones that surpassed the baseline.

Comparison of retrieval performance (Recall@5) across different planning strategies and datasets, highlighting the consistent gains from adding a reflection step.

Conclusion

InteGround provides a rigorous benchmark for complex retrieval and verification tasks. Our results show that while LLMs struggle with the rationalization trap during verification, their retrieval performance can be significantly bolstered through logic-based planning and iterative reflection. Future work will explore multi-modal grounding and the integration of symbolic planners to further enhance system reliability.

Summary updated

Abstract

Grounding large language models (LLMs) in external knowledge sources is a promising approach to ensuring faithful and accurate predictions. While existing grounding approaches work well for simple queries, many real-world information needs require synthesizing multiple pieces of evidence. We introduce "integrative grounding" — the challenge of retrieving and verifying multiple interdependent pieces of evidence to support a hypothesis query. To systematically study this problem, we repurpose data from four domains for evaluating integrative grounding capabilities. Our investigation reveals two critical findings: First, when verifying groundedness, while LLMs are robust to redundant evidence, they tend to rationalize using their internal knowledge when the provided grounding information is incomplete. Second, in examining retrieval planning strategies, we find that undirected planning can degrade performance through the introduction of noise, while premise abduction emerges as a promising approach due to its logical constraints. Additionally, we observe that LLMs’ zero-shot self-reflection capabilities consistently enhance grounding quality.
BibTeX
@misc{cheng2025integround,
  author = {Jay J. Cheng and Qianqian Zhuang and Haoran Li and Chunkit Chan and Xin Liu and Lin Qiu and Yangqiu Song},
  title = {{InteGround: On the Evaluation of Verification and Retrieval Planning in Integrative Grounding}},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.732},
}
Figure