Evidence-Bound Review of Oligogenic Candidates

Many patients with an obviously inherited disease leave the genetics clinic without a molecular answer. In one study of 47 families with cone-rod dystrophy, 78.72% of families stayed unexplained even after all 25 known genes were sequenced. Part of this gap comes from cases caused by two or more genes acting together, called digenic or oligogenic inheritance. Tools such as Hop now search a patient's exome for such variant pairs and return a ranked shortlist. A geneticist must then read the literature to decide whether any listed pair is biologically credible. Large language models (LLMs) could do that reading. In this setting, though, their best-known failure is also their most dangerous one: given two gene names, a fluent model will readily invent a shared pathway or an interaction that no paper supports.

We propose a reviewer for oligogenic shortlists that is only allowed to say what the literature says. For each candidate pair, it writes a short evidence dossier in which every claim is tied to a retrieved sentence published before a cutoff date, and each claim-to-sentence link is checked by a separate verifier. A learned rejector then hides pairs whose dossiers lack verified support. The central methodological commitment is a temporal holdout: the literature index is cut off before the date the causal pair was first reported. Without this, any literature-reading system "finds" the answer by recalling the very paper that discovered it. We expect, and will test, that unconstrained LLM reranking looks strong without the holdout and collapses under it. Whether evidence-bound review beats a cheap literature co-mention signal under the holdout is the question that decides whether the method is worth building.

Core Claim

The claim has two parts. First, among the top 50 pairs Hop returns, an evidence-bound reviewer with a learned rejector can cut the number of pairs a geneticist must read before reaching the causal one, at a fixed miss rate. It can do this while keeping the fraction of unsupported claims in its rationales near zero. Second, these gains survive a temporal holdout that removes the discovery paper and its descendants. Ungrounded LLM reranking will not survive it, because its apparent skill is largely memorization of published pairs.

The closest prior work leaves exactly this combination open, and each paper plays a different role here.

Two papers motivate the target. Hop scores variant pairs with a random-forest pathogenicity model (VarCoPP2.0) plus a random walk over BOCK, a 12-database gene–phenotype–disease graph. Hop never reads text and is biased toward well-studied genes. ORVAL offers similar scoring and digenic-effect labels as a web service, again with no literature reasoning and no written rationale.

Two papers bound the design from the other side. AIVARI Agent grounds an LLM in OMIM and ClinVar for whole-case variant reportability. It reasons strictly one gene at a time, so it cannot judge pairs. It also over-calls, with row precision of 0.351, and its authors name pruning false positives without losing sensitivity as unsolved. That is the problem our rejector targets, moved to the pair level. DUVEL shows that encoder models can detect gene-variant-gene-variant relations in 256-token fragments (F1 0.84 with BiomedBERT-large). It operates on text alone, with no patient, no phenotype and no generation. We reuse DUVEL as training and test data for our claim verifier.

Two papers supply methods. MedCite supplies the technique of breaking an answer into statements, re-retrieving evidence for each, and reranking citations. It also supplies the constraining observation that single-pass retrieval can make citation quality worse. Learning to Reject with a Fixed Predictor supplies the rejector: a separate model trained on top of a frozen predictor with a convex surrogate loss.

Finally, MHB constrains the design. It finds aggregate hallucination rates of 29.1% for Claude-4-Sonnet and 59.0% for Llama-4-Scout under adversarial clinical traps, and it finds models especially prone to inventing procedures and authorities. This is why we refuse to let the generator assert anything the verifier has not checked.

Research Program

The system consumes the top 50 pairs from an unmodified Hop run, along with the patient's phenotype written as Human Phenotype Ontology (HPO) terms, a standard vocabulary of clinical features.

For each pair, a retriever searches a PubMed index truncated at a cutoff date. It combines BM25 keyword search with MedCPT semantic search, following MedCite. Queries cover each gene alone, the two genes together, and each gene with the patient's phenotype terms.

An LLM then drafts a dossier of atomic claims. Allowed claim types are fixed in advance: gene A causes a phenotype overlapping the patient's; genes A and B act in the same complex or pathway; the two genes have been reported together in patients; and a functional assay exists for the specific variant. Each claim must quote a retrieved sentence.

A verifier then checks every claim against its quoted sentence and drops claims it cannot confirm. The verifier is a fine-tuned entailment model, with a DUVEL-trained relation head for co-occurrence claims. The dossier that reaches the clinician therefore contains only verified claims, and every dropped claim is logged so that we can measure the fabrication rate.

Last, a rejector decides whether to surface each pair, and the surfaced pairs are reranked. Its inputs are the Hop score, counts of verified claims by type, phenotype overlap, and the ratio of retrieved to verified claims. It is trained with the surrogate loss of Learning to Reject, with Hop plus dossier features as the frozen predictor.

Two design choices matter. The rejector never sees the LLM's own stated confidence, so it cannot be fooled by fluent certainty. And the generator never sees Hop's rank, so it cannot simply justify whatever pair is on top.

Baselines

The naive reference point is Hop's own ranking, used unchanged. Hop is the strongest existing method for this question because it was built and tested on exactly this task. Hop reports 71% top-50 accuracy and about 60% top-10 accuracy on 50,400 synthetic exomes, against about 23% for Exomiser and about 19% for OligoPVP. Our reviewer only reorders and filters Hop's top 50, so its recall ceiling is Hop's 71%. Beating Hop therefore means a higher top-10 and top-5 rate at an equal miss rate within the top 50, and fewer pairs read before the hit.

Hop's pathogenicity model is trained on OLIDA, the oligogenic disease database. We therefore retrain it with the test pairs held out and report that retrained Hop as the baseline. The original Hop numbers would contain the answer.

The strongest cheap literature baseline adds literature co-mention edges to Hop's knowledge graph, using PubTator gene co-occurrence counts before the cutoff. We then rerun the random walk, or alternatively rerank with a logistic model on co-mention counts. Both versions see the same truncated index and the same top 50 lists, and each gets the same tuning budget on the same validation pairs.

A second literature baseline runs the DUVEL BiomedBERT-large extractor over the truncated index. It promotes any pair the extractor finds asserted in text. No reported result was found on Lacuna for either literature baseline applied on top of Hop.

The ungrounded LLM baseline gives the same LLM the pair list and the phenotype and asks it to rerank them from its own knowledge. We run it with and without retrieval: the retrieval version is plain single-pass retrieval with no claim verification. No reported result was found on Lacuna for LLM reranking of oligogenic shortlists.

Three ablations isolate the key ingredients. The first removes the verifier and keeps every generated claim. The second replaces the learned rejector with a threshold on the LLM's stated confidence. The third uses only the verified-claim counts, with no learned rejection.

For the rejector itself, the reference point is the result reported in Learning to Reject with a Fixed Predictor. On their task, the surrogate-loss rejector kept 90.6% coverage while halving errors, against about 64.4% for MaxProb, which rejects whenever the model's top output probability is low. We ask whether a comparable gap appears on our task.

Why not just…?

Why not just let an LLM rerank Hop's list from its own knowledge? On published pairs it will likely do well, because OLIDA pairs come from papers the model has probably read. That is precisely why this baseline is dangerous: it measures recall of known answers, not the ability to diagnose. The temporal holdout settles the question. If ungrounded reranking keeps its advantage when the model's training data postdates the pair, which we probe by restricting the evaluation to pairs published after the model's training cutoff, then it is a real competitor and we will say so. We also count fabricated interactions in its rationales, using the same verifier, because MHB shows that a strong ranking can coexist with high rates of invented claims.

Why not just add literature co-mention edges to Hop's graph? This is the most serious threat. It is cheap and deterministic, and it captures much of what a dossier would say. Under the holdout, co-mention of the causal pair itself is mostly gone by construction, so any gain from the dossier must come from verified links between each gene's function and the patient's phenotype. We set the decision rule in advance. If the co-mention reranker comes within the confidence interval of the full system on top-10 rate and on pairs read, we drop the LLM generator. The surviving contribution would then be the temporal-holdout benchmark plus a co-mention rejector, and we will report that result.

Why not just threshold the model's confidence instead of training a rejector? The confidence-threshold ablation tests this directly. Prior evidence from Learning to Reject with a Fixed Predictor suggests that simple thresholds over-reject, keeping about 64.4% coverage versus 90.6% for the trained rejector at matched error. That evidence comes from a different task, however, and with only 50 candidates per case the gap may shrink. If it does, we will use the simpler threshold.

Proposed Experiments

The main benchmark regenerates Hop's synthetic-exome protocol with the released Hop codebase. OLIDA pairs are spiked into healthy 1000 Genomes and UK10K exomes, with phenotypes taken from each pair's source report. For each pair, the literature cutoff is set to the year before its first OLIDA citation. Train, validation and test splits are made by gene, so that no test gene appears in training. We also report how much of BOCK itself postdates the cutoff, since that is leakage present in every baseline.

The metrics are top-k accuracy for k of 1, 5 and 10; the expected number of pairs read before the hit; the miss rate among surfaced pairs; and the claim fabrication rate before and after verification. Two physicians audit a sample of dossiers for support, following the expert-agreement check reported for MedCite.

Component checks precede the full pipeline. The verifier's relation head must match DUVEL's reported F1 of 0.84 on DUVEL's own split before we trust it inside the pipeline.

Four checks decide the outcome. The first is the gap between performance with and without the temporal holdout for every literature-using method. We predict a large drop for ungrounded reranking and a small drop for evidence-bound review. The second is the full system against the co-mention reranker under the holdout; this is the go or no-go test for the LLM component. The third is the verifier ablation's effect on fabricated claims at equal ranking quality. The fourth breaks results down by gene annotation depth, to test whether literature review worsens or eases Hop's bias toward well-studied genes.

As a qualitative real-data case, we will replay the hydrocephalus cohort of Congenital hydrocephalus: new Mendelian mutations and evidence for oligogenic inheritance, with the index cut off before that paper. We then ask whether the IFT172–TTC21B combination is surfaced, and with what verified evidence.