Evidence Interfaces Shape How Retrieval-Augmented Readers Use Support

Junchi Liao, Jiawen Deng, Fuji RenarXiv 2026 · arXiv

TL;DR

Standard retrieval-augmented generation (RAG) evaluations often conflate retrieval failures with reader formatting issues, making it hard to diagnose why a system underperforms. By introducing the concept of "evidence interfaces" and tracking whether the entire multi-hop reasoning chain survives truncation, this paper reveals that aggregate top-$k$ metrics mask drastically different reader behaviors.

What Problem Does This Paper Address?

In Retrieval-Augmented Generation (RAG), a retriever fetches relevant documents, and a reader (typically an LLM) uses them to generate an answer. When a RAG system fails, developers usually look at aggregate metrics like top-$k$ F1 scores. However, a low score is highly ambiguous:

  1. Availability Failure: Did the retriever fail to find a crucial piece of the evidence chain?
  2. Usability Failure: Is the evidence present, but formatted or ordered in a way that the reader model struggles to process?

This paper tackles this measurement problem. Without separating these two failure modes, system designers risk optimizing the wrong component—such as swapping out a retriever when the reader simply needs a better prompt layout, or vice versa.

Methods

The authors formalize the reader-facing presentation of text as an evidence interface—a renderer that selects, orders, and structures evidence before it hits the reader.

To cleanly separate availability from usability, they introduce a diagnostic protocol:

  • Matched Adaptation: Instead of just swapping prompts at inference time, they train separate reader models (using parameter-efficient LoRA adapters) on specific evidence interfaces. This ensures the reader is fully adapted to the format it is evaluated on.
  • Complete-Support Recall (CSR@k): A strict metric that evaluates whether every single unit of a multi-hop evidence chain is present in the top-$k$ window.
  • Coverage-Conditioned Analysis: They split the evaluation dataset into two bins: one where the full support chain survives truncation, and one where it is broken.

The framework compares several interface families, ranging from raw un-ordered context to "support-first" layouts (moving gold evidence to the front) and oracle text.

Overview of evidence-interface diagnostics separating availability from usability.

Datasets

The study evaluates RAG performance on three popular multi-hop question-answering benchmarks that feature explicit sentence- or paragraph-level support annotations:

  • HotpotQA
  • 2WikiMultiHopQA
  • MuSiQue

These datasets are ideal because answering a question correctly requires connecting multiple distinct pieces of information scattered across distractor texts.

Key Insights and Ideas

  • The Top-$k$ Blind Spot: Aggregate top-$k$ scores average out two completely opposite phenomena. When the full support chain survives in a short window, the compression actually helps the reader. When even one link is missing, performance plummets.
  • Multi-Hop is All-or-Nothing: Standard retrieval evaluations often use "mean support fraction" (giving partial credit for finding some documents). The authors show this is deceptive for multi-hop QA: missing the final hop makes the question completely unanswerable, meaning partial retrieval is often as bad as zero retrieval.
  • Layout Dictates Learning: How evidence is ordered during training fundamentally changes what the reader learns. Moving the necessary support to the beginning of the prompt ("support-first") consistently yields better-adapted readers, proving that "lost-in-the-middle" effects are a critical training-time factor, not just an inference quirk.

Key Results

The authors split 1,000 validation examples by whether a ranker-specific top-5 (@5) window captured the full support chain:

  • The Split Effect: On the MuSiQue dataset, evaluating the @5 window as an aggregate score shows a massive loss of -0.132 F1 compared to raw context. However, when conditioned on coverage, the @5 window actually improves F1 by +0.065 when full support is present, and drops by -0.222 when support is missing.
  • Support-First Headroom: Simply moving the gold support paragraphs to the front of the text (SF) while keeping distractors improved F1 across all datasets (+0.026 on HotpotQA, +0.039 on 2Wiki, and +0.070 on MuSiQue), demonstrating significant reader-side headroom.
Figure 2: Main evidence-interface effects. (a) Matched Qwen2.5-Coder-14B F1 across interface families; gold-chain triples are available only for 2Wiki and form an answer-exposing ceiling. (b) On the expanded 1000-example answer evaluation, the ranker-specific @5 losses are small when annotated support is selected and large when support is missing. (c) Support-first gains are positive across available reader conditions.

Limitations and Future Work

  • Annotation Dependency: The diagnostic framework relies entirely on datasets with gold support annotations. In many real-world production RAG pipelines, such fine-grained labels do not exist.
  • Closed Candidate Pools: The study focuses on re-ranking and formatting within a fixed, closed pool of documents rather than open-web retrieval.
  • Future Direction: The authors suggest moving away from gold-annotated diagnostics toward training learned interface builders—components explicitly optimized to arrange and compress text in ways that maximize usability for a specific target reader model.

Summary updated

Abstract

In multi-hop RAG evaluation, a top-k answer score can hide two different failures: the retrieval window may drop part of the support chain, or it may contain support in a form the adapted reader does not use well. We call this reader-facing form of retrieved evidence an evidence interface. Using three support-annotated multi-hop QA benchmarks, we compare matched adapted readers trained with raw context, retrieval windows, and gold-support diagnostic renderings. These comparisons distinguish support-availability failures from remaining reader-interface effects. Top-k windows become interpretable only after checking whether the complete annotated support chain survives: when it does, short ranked windows can match or improve over raw context; when it does not, missing support explains much of the loss. Gold support-first improves matched readers; on 2Wiki and MuSiQue, a support-supervised ranker raises coverage and recovers raw-context quality at lower prompt cost, while retaining gold headroom. Support-removal checks further show that the gains rely on exposed evidence, not only answer priors. On support-annotated evaluations, top-k answer scores should therefore be reported together with complete-support coverage.
BibTeX
@misc{liao2026evidence,
  author = {Junchi Liao and Jiawen Deng and Fuji Ren},
  title = {{Evidence Interfaces Shape How Retrieval-Augmented Readers Use Support}},
  year = {2026},
  eprint = {2607.17108},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2607.17108},
}
Figure