CLADE: Category-Level Answering from Deficient Evidence in Retrieval-Augmented Language Models

S M Rayeed, Charles V. Stewart, Yao MaarXiv 2026 · arXiv

TL;DR

Current Retrieval-Augmented Generation (RAG) systems struggle with "resolvable insufficiency," where retrieved text is accurate but too broad—for example, naming a disease family instead of a specific condition. Instead of abstaining, Large Language Models (LLMs) often take this coarse evidence as a license to guess, a phenomenon the authors call CLADE (Category-Level Answering from Deficient Evidence), which significantly degrades precision.

What Problem Does This Paper Address?

In RAG, we typically view context sufficiency as binary: either the retrieved passage contains the answer, or it doesn't. However, real-world documents often describe facts at a coarser level of granularity than a user's question requires. If you ask, "What specific disease does lovastatin treat?" and the retriever returns a passage saying it treats "hyperlipidemia" (the broad family), the model faces a dilemma.

The authors identify this as resolvable insufficiency. The text alone cannot settle the fine-grained answer, but the category it mentions points to a finite set of candidates in a domain ontology (like a medical database). Ideally, a model should recognize this gap and consult the ontology. Instead, models often exhibit CLADE: they stop abstaining and confidently guess a specific member of that category, often getting it wrong.

Methods

The researchers systematically investigated CLADE through four main experiments:

  1. Abstention Evaluation: They compared model behavior across three types of passages: named (contains the exact answer), family-only (contains only the broader category), and silent (mentions the subject but not the answer or its family).
  2. Counterfactual Evaluation: To isolate the effect of the category name, they performed "page edits." They took passages that originally named the answer and either replaced the answer with its parent category name or deleted the sentence entirely.
  3. Testing "Simple Fixes": They evaluated whether common techniques—like confidence filtering, scaling to larger models (up to 14B parameters), or explicitly instructing the model to abstain on broad categories—could mitigate the guessing behavior.
  4. Ontology Recovery: They designed a neuro -symbolic pipeline to see if an ontology could turn a coarse mention into an exact answer by extracting the family name, listing its members, and using various "choosers" (like popularity priors or PubMed co-mentions) to pick the right one.
Figure 1: Resolvable insufficiency as an agent workflow, with the bottleneck we measure at each stage. The retrieved passage is accurate and relevant but names only the answer’s family. A page-type check should send it to an ontology, which expands the family into a short candidate list for a chooser. The dashed red path is Clade: the model guesses from the family-only passage instead. In current practice, only the ontology step works reliably.

Datasets

The study focused on two high-stakes domains where hierarchical knowledge is critical:

  • Biomedical: Drug-disease relations from PrimeKG, linked to the MONDO Disease Ontology. They used Wikipedia articles and PubMed abstracts as the text source.
  • Ecological: Species-to-species interactions (e.g., "what does this butterfly eat?") from GloBI, using biological taxonomy (genus, family) as the hierarchy and Wikipedia for text.

Key Insights and Ideas

  • Coarse Evidence is a "License to Guess": A "silent" page correctly makes models realize they don't know the answer. However, adding just a family name (e.g., "hyperlipidemia") makes models stop saying "I don't know" and start guessing, even though the text provides no evidence for which specific subtype is correct.
  • The "Confidence Trap": Models are often just as confident in their wrong guesses from coarse evidence as they are in their correct answers from named evidence. This makes standard confidence-based filtering ineffective.
  • Scaling Isn't the Solution: Moving from a 0.5B to a 14B model improves performance on clear evidence but does not stop the model from guessing incorrectly when evidence is coarse.
  • The Bottleneck is "Finding the Family": While ontologies are great at listing candidates once a family is identified, automatically extracting the correct family from a noisy Wikipedia page is surprisingly difficult. String matching often fails because pages mention many unrelated categories (side effects, comorbidities, etc.).

Key Results

  • Abstention Drop: On biomedical questions, family-only passages reduced model abstention by 13.8 percentage points compared to silent passages.
  • Precision Collapse: While models achieve 84–95% precision when the exact name is in the text, precision falls to 45–59% on family-only passages. In the ecological domain, it drops even further to near-chance levels (22–28%).
  • Counterfactual Impact: Replacing a specific disease name with its family name lowered abstention by 17.4 points compared to deleting the information entirely.
  • Ontology Success: If the correct family is provided, an ontology list contains the true answer 98.5% of the time. Using an LLM to pick the family from the page and then using a popularity prior to choose the specific entity raised end-to-end recovery from 10.0% to 26.8%.
Figure 3: Results of counterfactual edits, showing how keeping a family name suppresses abstention compared to removing the sentence.

Limitations and Future Work

The authors acknowledge several limitations:

  • Multiple Choice vs. Free Text: Most experiments used multiple-choice questions. While free-text checks showed similar patterns (models often just output the family name as the answer), the dynamics of open-ended generation deserve more study.
  • Domain Specificity: The work is limited to medicine and ecology. While these are prime examples of hierarchical domains, the "CLADE" effect likely exists in other areas like law or technical support.
  • The "Hand-off" Problem: The biggest remaining challenge is reliably identifying when a model should stop reading and start querying an ontology. Future work should focus on better "checkers" that can distinguish between a specific entity mention and a category-level mention in context.

Summary updated

Abstract

Retrieval-augmented systems often retrieve passages that are accurate and relevant but state the answer at a coarser level of granularity than the question requires. For example, a question may ask which specific disease a drug treats (familial hyperlipidemia), while the retrieved passage mentions the broader disease family (hyperlipidemia). Prior work on context sufficiency treats such coarse passages simply as insufficient. We argue that they form a distinct case, which we call resolvable insufficiency: text alone cannot settle the fine-grained answer, but the broader category it mentions maps to a short list of candidate answers in a domain ontology. When language models read these coarse, family-only passages, they exhibit a phenomenon we call Clade (Category-Level Answering from Deficient Evidence): they answer instead of abstaining or searching further. On biomedical questions from PrimeKG, family-only passages reduce abstention by 13.8 points compared to uninformative passages, and the resulting guesses are wrong about half the time. A counterfactual test confirms the cause: replacing an exact disease name with its broader family name lowers abstention by 17.4 points compared to deleting the sentence. Simple fixes like confidence filtering, scaling to a larger model, and explicitly instructing the model to abstain cannot resolve Clade, and this replicates on ecological species interactions in GloBI. Finally, we test whether an ontology can turn coarse passages into exact answers. Given the correct family, an ontology list includes the true answer in 98.5% of cases, but string matching finds that family on the page for only 19.8%. Letting the model choose the family raises this to 30.0% and end-to-end recovery from 10.0% to 26.8%. Overall, our results show when language models should stop guessing and consult an ontology, and highlight which steps of that process remain unsolved.
BibTeX
@misc{rayeed2026clade,
  author = {S M Rayeed and Charles V. Stewart and Yao Ma},
  title = {{CLADE: Category-Level Answering from Deficient Evidence in Retrieval-Augmented Language Models}},
  year = {2026},
  eprint = {2611.14355},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.14355},
}
Figure