When a crop variety resists a pathogen, the explanation often depends on which copy of a gene it carries, when that gene switches on, and in which tissue. Two versions (alleles) of the same tomato gene that differ by 13 amino acids can have opposite effects: one confers resistance to powdery mildew and the other increases susceptibility. Geneticists partly attribute "missing heritability", the share of resistance that genome-wide mapping cannot explain, to methods that average over such distinctions. Language-model systems that answer questions by chaining several retrieval steps make the same mistake. They join evidence across documents on a shared gene name, so a chain can pick up a fact about the resistant allele in one step and a fact about the susceptible allele in the next, and produce a confident, fluent, wrong mechanism.
This note proposes to measure that failure and to fix it. The fix is to treat each intermediate entity in a reasoning chain as a qualified record (gene plus allele or haplotype, host genotype, time after infection, tissue) rather than a bare name. Every later retrieval step must respect those qualifiers, and the chain branches whenever the evidence splits across them. The plant-resistance literature is the realistic test bed because qualifier collapse is the central problem there, not a corner case. A controlled counterfactual version of a standard multi-hop benchmark lets us test the same failure at scale.
Core Claim
Multi-hop retrieval-augmented generation is a language model that alternates between writing reasoning steps and fetching documents. Its gap is usually described as a recall problem: a single retrieval pass cannot find evidence for step two because the query does not yet mention the intermediate "bridge" entity found in step one. Interleaved and iterative methods close much of this recall gap. We claim a second, unmeasured failure appears once recall is solved. The bridge entity is resolved to the right name but the wrong referent, and downstream evidence is joined across incompatible biological contexts.
Our hypothesis is that current multi-hop systems will score well on single questions while failing contrast pairs. A contrast pair is two questions that differ only in an allele, time point, or genotype and have different correct answers. Carrying explicit qualifiers through each hop, with retrieval that must match them and branching when they conflict, should close most of that contrast gap without hurting standard benchmarks. The new part is the failure mode and its measurement: same-name, different-referent joins. Neither multi-hop retrieval nor plant-genomics work has measured it, and it maps directly onto how unphased genetic analysis hides allele-specific effects.
Grounding in Prior Work
The biological motivation comes from three papers that each show a single gene name hiding opposing or conditional effects. In ShORR-1, a Novel Tomato Gene, Confers Enhanced Host Resistance to Oidium neolycopersici, the authors cloned resistant and susceptible alleles of one gene and showed with overexpression and silencing that they act in opposite directions. In A chromosome-level phased genome enabling allele-level studies in sweet orange: a case study on citrus Huanglongbing tolerance, the authors built a reference genome that separates the two parental copies and found that 32.5% of genes show biased expression between them. They also showed that a translocation in a tolerant mutant upregulates heat-shock genes in an allele-specific way. In Transcriptome Analysis and Functional Characterization of the HvLRR_8-1 Gene Involved in Barley Resistance to Pyrenophora graminea, the authors profiled an immune and a susceptible barley variety at 7, 14 and 18 days after inoculation, showing that the timing of defense activation separates the two. In all three cases a claim about "the gene" is meaningless without its qualifier. These papers motivate the work, and they also supply gold facts for the benchmark.
The method builds on the iterative retrieval line. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions (IRCoT) uses each reasoning sentence as the next query. Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy (ITER-RETGEN) feeds the whole previous answer back as the query. Chain-of-Retrieval Augmented Generation (CoRAG) trains models on rejection-sampled retrieval chains. All three resolve bridges by name and say nothing about whether two passages mentioning the same name refer to the same thing. We keep their loop and change what is passed between steps.
GRAT: Guiding Retrieval-Augmented Reasoning through Process Rewards Tree Search supplies the idea of branching and scoring alternative chains with tree search. It branches on alternative sub-questions, whereas we branch on qualifier conflicts in the evidence, which is a cheaper and more targeted trigger. The closest existing system is GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation. It combines text retrieval with subject-predicate-object queries over a knowledge graph and reports evaluations including agriculture. It assumes the graph's nodes are already correctly individuated, and that is exactly the step where allele collapse happens when a graph is extracted from papers. We treat it as the strongest competitor, not as prior art to dismiss.
The benchmark method comes from ♫ MuSiQue: Multihop Questions via Single-hop Question Composition. MuSiQue builds multi-hop questions by composing single-hop ones and filtering out questions that can be answered by shortcut. We reuse that composition procedure and add contrast twins. PubMedQA: A Dataset for Biomedical Research Question Answering constrains the design. It found that roughly 21% of abstracts state results only as raw statistics, so many qualifiers (fold changes, time points) appear only in numbers. Our qualifier extractor must therefore read tables and figure captions, not just prose.
Research Program
The first artifact is a benchmark we call AlleleHop. It has two parts.
The realistic part is built from plant-pathology papers and structured resources. These include the phased sweet orange genome with its 740 transcriptomes and its tolerant (T19) and sensitive (T78) mutants, the barley time-course RNA-seq under the accessions listed in the HvLRR_8-1 paper, the ShORR-1 allele constructs, and the faba bean root and rhizosphere data in Integrated metabolomics and metagenomics uncover pathogenic mechanisms of Fusarium wilt and faba bean defense responses. Plant pathologists write single-hop facts with explicit qualifiers. We compose those facts into two- to four-hop questions. Every question gets a twin that flips one qualifier and changes the answer. An example pair asks what the overexpressed G allele does to salicylic-acid marker genes, and then what the overexpressed M allele does to host susceptibility. The realistic part will be small, in the hundreds of question pairs, because expert annotation is the bottleneck.
The controlled part therefore takes MuSiQue and 2WikiMultiHopQA questions and injects counterfactual twin documents for the bridge entity. Each twin shares the entity's name but carries a different qualifier, such as a different year, role, or edition, and implies a different downstream answer. This turns name-versus-referent confusion into a variable we can vary at scale and outside biology.
The second artifact is the method, qualified-bridge retrieval. At each hop the model emits the answer to the sub-question as a record: a name plus a small set of typed qualifiers extracted from the supporting passage. The next retrieval is scored on two things: text similarity, and agreement between the record's qualifiers and qualifiers extracted from each candidate passage. A passage whose qualifiers contradict the record cannot serve as support, though it can still be shown to the model as labeled contrast. If retrieved evidence splits across qualifier values the question has not fixed, the chain forks. Each branch is carried to an answer with a one-step rollout in the style of GRAT. The final answer either picks the branch the question specifies or reports the conditional answer.
Training follows CoRAG's rejection sampling. Only chains whose cited evidence respects qualifiers are kept, so the model learns to emit records rather than names. We will run it on an 8B open-weight model so the numbers sit next to CoRAG-8B and GRAT's Llama3-8B results.
Evaluation uses two metrics beyond standard exact match and F1. Contrast consistency is the fraction of twin pairs where both members are answered correctly. Qualifier leak rate is the fraction of answered questions whose cited supporting passages include one whose gold qualifier contradicts the question's.
Baselines
The naive reference point is single-pass dense retrieval: retrieve the top documents for the original question and answer in one shot. Chain-of-Retrieval Augmented Generation reports single-step retriever recall of 29.0% on MuSiQue. It sets the floor and shows how much of any contrast failure is just recall failure.
IRCoT and ITER-RETGEN are the standard iterative baselines. They will be run with the same retriever, corpus, base model and number of retrieval calls as our method. ITER-RETGEN will be capped at the two to three iterations where its authors report convergence.
CoRAG is the strongest trained chain method. It reports 47.1% retriever recall on MuSiQue and an advantage of over 10 exact-match points over IterDRAG and Search-o1-32B. We will retrain it on the same rejection-sampled data budget as our method, with qualifier filtering removed, so the only difference is the qualified record.
GRAT is the strongest search method. It reports 75.2% accuracy on 2WikiMultiHopQA with Llama3-70B, and 66.8% with Llama3-8B after self-training (up from 60.6%). It will receive a matched token budget so that our branching is not credited for spending more compute.
GraphSearch is the strongest structured competitor. It reports MuSiQue substring exact match of 51.00 against 35.00 for LightRAG. It will be run twice: once on a knowledge graph extracted from the papers by an LLM, and once on a gold graph whose nodes are hand-split by allele, genotype and time point.
On AlleleHop itself there is no reported result on Lacuna for any of these systems, since the benchmark is new. All numbers there will come from our own matched runs.
Beating the baselines means two things. First, a clear gain in contrast consistency and a clear drop in qualifier leak rate over matched CoRAG and GRAT on both parts of AlleleHop. Second, staying within noise of their published-scale numbers on unmodified MuSiQue and 2WikiMultiHopQA.
Three ablations isolate the key ingredient. The first passes the extracted qualifiers in the prompt only, with no filtering at retrieval time. The second applies filtering without branching. The third replaces our extracted qualifiers with standard vector-database metadata filters set from the question.
Why not just…?
Why not just prompt a stronger model to keep track of alleles? This is a real possibility, and the prompt-only ablation tests it directly with the same qualifier information our method uses. Our expectation comes from the error-compounding results in CoRAG and GRAT: once a wrong-referent passage enters the context, later steps condition on it. Prompting asks the model to ignore evidence it has already been shown. Filtering keeps that evidence from being used as support. If prompt-only matches full filtering on contrast consistency, the contribution shrinks to the benchmark and the measurement, and we will report it that way.
Why not just use metadata filters in the vector database? Filters work only when every document carries the qualifier as clean metadata and the question states it up front. In multi-hop questions the qualifier that matters is often discovered mid-chain; the time point, for instance, comes from the step-one passage. Metadata also has to be extracted from prose and statistics, which PubMedQA shows are often uninterpreted. The metadata-filter ablation measures how much of the gain these simple filters capture.
Why not just build an allele-level knowledge graph and run GraphSearch on it? This is the most serious threat. If GraphSearch on the gold, hand-split graph matches our method, then the reasoning loop is not the bottleneck and the real problem is graph construction. Our claim would then reduce to "extract qualifiers before you build the graph", which is still useful but smaller. The two-graph comparison settles it. The size of the gap between GraphSearch on the extracted graph and on the gold graph is itself the measure of how much collapse occurs at extraction.
Why not just solve missing heritability with better statistics, such as epistasis models or phased mapping? That is the right tool for estimating genetic effects, and this proposal does not compete with it. Our target is the downstream step where researchers and tools combine published results into mechanistic chains, and that is where collapse reappears even after phasing has been done, as in the sweet orange work.
Proposed Experiments
The first decisive check is diagnostic and needs no new method. We run IRCoT, ITER-RETGEN, CoRAG and GRAT on controlled AlleleHop, sweeping the number of injected same-name twins from zero upward. If contrast consistency stays flat as twins are added, the hypothesized failure does not exist and the project stops there. If it falls while single-question accuracy holds, the failure is real and invisible to current metrics.
The second check is the method comparison on both parts of AlleleHop, with the baselines and ablations above at matched compute. It is reported separately by qualifier type (allele, genotype, time point, tissue) to see which distinctions current systems collapse most.
The third check is a no-regression test on unmodified MuSiQue and 2WikiMultiHopQA. It confirms that the added qualifier machinery does not cost the recall gains that iterative methods earned.
The final check is a small retrospective case study, framed as illustrative rather than statistically powered. We remove the HvLRR_8-1 functional paper and the T19 translocation analysis from the corpus. We then ask each system to rank candidate genes from the remaining evidence: the 20 resistance genes among barley's 457 immunity-associated differentially expressed genes, and the heat-shock genes in sweet orange. We check whether qualifier-aware chains rank the validated genes higher than name-joined chains do. A positive result would suggest the literature-level analogue of missing heritability is recoverable. A null result would be reported as such.
References
- ShORR-1, a Novel Tomato Gene, Confers Enhanced Host Resistance to Oidium neolycopersici
- A chromosome-level phased genome enabling allele-level studies in sweet orange: a case study on citrus Huanglongbing tolerance
- Transcriptome Analysis and Functional Characterization of the HvLRR_8-1 Gene Involved in Barley Resistance to Pyrenophora graminea
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
- Chain-of-Retrieval Augmented Generation
- GRAT: Guiding Retrieval-Augmented Reasoning through Process Rewards Tree Search
- GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- PubMedQA: A Dataset for Biomedical Research Question Answering
- Integrated metabolomics and metagenomics uncover pathogenic mechanisms of Fusarium wilt and faba bean defense responses