Checkable Claims First: Verifier-Routed Agents for Tumor Board Recommendations

Genomic tests now routinely find a drug target in cancer patients, yet most of those patients never receive the drug. At the Munich molecular tumor board, a panel of oncologists, pathologists and geneticists that turns sequencing results into treatment advice, 35.3% of biliary tract cancer patients had an actionable alteration and 46.4% received a recommendation, but only 19.4% received the recommended therapy. Rapid physical decline and logistical delays were the main reasons (A Retrospective Analysis of Biliary Tract Cancer Patients Presented to the Molecular Tumor Board at the Comprehensive Cancer Center Munich). Patients who did receive targeted therapy had a median overall survival of 19 months, against 8 months for those who were recommended it but never got it. So the pipeline breaks less at "which variant matters" and more at the slow, manual step of assembling a recommendation that can be checked and acted on. Language-model agent systems are an obvious candidate for automating that step, but in medicine their weakest point is that one agent's error gets accepted by the next as fact.

We propose to split a tumor board recommendation into small, explicit claims and send each claim to the cheapest checker that can settle it. Some claims are settled by deterministic lookups, such as "this variant has OncoKB evidence level 1 for drug X" or "this trial is recruiting, includes this histology, and excludes ECOG ≥2". The others go to a language-model critic, or are flagged for a human when no checker can verify them. The output is a ledger of claims, each marked as machine-verified, model-judged, or needing clinician review. Our hypothesis is that in precision oncology, splitting a task across agents helps mainly because it separates checkable from uncheckable sub-claims. Role-play debate between agents, on this view, contributes little. If that holds, verifier-routed decomposition should match or beat deep agent debate on accuracy, propagate far fewer upstream errors, and shrink the number of claims a busy clinician must personally check. That human checking time, not model inference time, is the turnaround cost that matters.

Core Claim and How It Differs

Recent work applies agents to pieces of this workflow. nMAS routes oncology feature extraction through three tiers: deterministic code, small models and a large model. It reaches a rank-weighted F1 of 85.0% against 66.4% for a single universal-abstraction baseline, but its validation agent only checks that a value appears verbatim in the source, not that it is clinically true (From Analytics to Tumor Boards: An Evidence-Linked Multi-Agent Workflow for Oncology Feature Extraction). We reuse its idea of routing by difficulty and move it downstream, from extraction to the recommendation itself. MALADE pairs each pharmacovigilance agent with a language-model critic. The critic corrected 44.5% of category-level outputs, and the system reached an AUC of 0.90 on the OMOP adverse-event ground truth (MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilance). The Adaptive Memory and Reflection system (AMR) mimics a clinical board on multiple-choice exams and reaches 93.2% on MedQA (Adaptive Memory and Reflection Multi-Agent System for Medical Question Answering). Both rely on model-on-model checking. That is exactly the mechanism through which errors cascade, a failure mode the multi-agent survey names as a central open problem (A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges). The closest system to ours is the GPT-4 oncology agent that calls vision models, MedSAM, a RECIST calculator and the OncoKB interface. It reaches 87.2% decision completeness against 30.3% for GPT-4 alone, but its citation accuracy is only 75.5% (Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology). It uses tools to generate answers. We use deterministic sources mainly to verify claims, and we make verification status part of the output.

The rule-based tradition plays two roles here. It is a constraint and it supplies part of the method. HL7 Arden Syntax medical logic modules, which are small executable if-then rules, are auditable. They break down, however, under ambiguous and fast-changing genomic evidence and they produce alert fatigue (Towards an interoperable information infrastructure providing decision support for genomic medicine). Our checkers are such rules, but they apply only to claims that the agents have already reduced to rule-checkable form. They do not have to cover the whole decision.

Research Program

The system has three parts. First, an extraction stage reuses nMAS-style tiering to produce a structured patient record: variants, histology, prior lines of therapy, performance status, labs and organ function. Second, a decomposition agent writes a candidate recommendation as typed claims. The claim types are variant-to-drug evidence, resistance or contraindication, trial eligibility per criterion, and timing feasibility (whether the drug or trial can realistically start within the patient's expected window). A separate condition-mining agent, modeled on MACM's division into a Thinker, a Judge and an Executor (MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems), proposes non-obvious options such as pathway-level or repurposing candidates, in the style of the inflammatory bowel disease CMap analysis (Computational drug repurposing for inflammatory bowel disease using genetic information). Third, a router sends each claim to a checker. Evidence-level claims go to an OncoKB or guideline lookup. Trial claims use exact metadata filtering followed by per-criterion reasoning, following the hybrid design that found pure embedding search confuses small-cell and non-small-cell lung cancer (End-To-End Clinical Trial Matching with Large Language Models). Arithmetic claims go to code. Only the leftover judgment claims go to a language-model critic or to the human queue. A claim that fails its check is sent back to the agent that wrote it, along with the checker's output.

The analysis centers on error propagation. We define the cascade rate as the fraction of upstream errors that survive into the final recommendation. Upstream errors include a wrong variant nomenclature, a wrong histology, or a missed prior-therapy line, and they are either naturally occurring or deliberately injected. We measure the cascade rate separately for each claim type and each checker type.

Baselines

The naive reference point is a single frontier model given the structured patient record and a detailed chain-of-thought prompt. It shows how much any structure adds. In the oncology-agent study, GPT-4 alone reached only 30.3% completeness on 20 simulated gastrointestinal cases (Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology).

The strongest existing method is the single tool-orchestrating oncology agent from that same study, which reports 87.2% decision completeness, 91.0% clinical correctness and 75.5% citation accuracy. It is the strongest competitor because it already has access to the same deterministic sources we use, through OncoKB and guideline retrieval. Any gain we report must therefore come from how the sources are used, not from having them. We will re-implement it with the same base model, the same tools, the same guideline corpus and the same token budget per case. Prompts for both systems get an equal tuning budget on a held-out development split. Beating it means a lower cascade rate and higher claim-level correctness at no worse completeness, with a citation or evidence accuracy clearly above its 75.5%.

For trial matching, the comparator is the hybrid matching pipeline, which reports 93.3% top-10 retrieval and 88.0% per-criterion accuracy before human re-adjudication, rising to 92.7% after (End-To-End Clinical Trial Matching with Large Language Models). We will run it unchanged on our cases.

A language-model-critic debate baseline in the AMR and MALADE style uses the same decomposition into claims, but every claim is checked by a model critic. It is matched to our system on tokens. The key ablation removes the deterministic checkers while keeping everything else, and a second ablation removes the decomposition into claims while keeping the checkers available as tools. Together these two ablations show which ingredient carries the effect.

Finally, a deterministic baseline turns OncoKB levels and trial filters into Arden-style rules with no language model, which shows how much of actionability is pure lookup.

Why not just…?

Why not just use the single tool-calling agent, which already reaches 87.2%? This is the real threat. Its tools may already give most of the verification benefit we claim. The second ablation tests this directly: if a single agent with the same checkers available as tools matches our cascade rate and the size of our human-review queue, then the decomposition into claims adds nothing, and we will report that. Our reason to expect a difference is that a single orchestrator decides for itself when to verify. Its 75.5% citation accuracy suggests it often does not.

Why not just run one model several times with self-critique, using the same token budget? Self-collaboration found that a single model checking its own code shows confirmation bias, and that a separate Tester role helps: a 29.9% relative Pass@1 gain on HumanEval (Self-collaboration Code Generation via ChatGPT). That evidence comes from code, where tests are executable, which is arguably our very point. We include a token-matched self-consistency baseline rather than assume the result transfers to oncology.

Why not just use rules, since actionability is mostly an OncoKB lookup? For tier-1 variant-to-drug pairs, rules probably are enough, and we expect the rule baseline to tie on those claims. The attrition data argue that the gap lies elsewhere: in contraindications, in trial eligibility written as narrative text, and in timing feasibility. Static rules handle those poorly (Towards an interoperable information infrastructure providing decision support for genomic medicine). The per-claim-type breakdown will show where rules suffice.

Why worry about agent latency at all? We mostly do not. Model calls take minutes, while the TuPro loop needed four weeks to administer 87% of its recommendations (Feasibility of multiomics tumor profiling for guiding treatment of melanoma). We therefore measure turnaround as the number of claims, and the minutes, a clinician must review. We do not treat token latency as a meaningful cost here.

Proposed Experiments

No open benchmark of tumor board deliberations exists, so the primary dataset must be built. With a partnering board, we will take de-identified retrospective cases with the panels, notes and actual board decisions. Oncologists will adjudicate the claims in each case, and records will show whether each recommendation was received and, if not, why. The Munich-style annotation of reasons for non-receipt directly supplies labels for the timing-feasibility claims. Public components cover subtasks. The first is the n2c2 2018 cohort-selection challenge for per-criterion eligibility. Zero-shot GPT-4 beat the prior best by 6 Macro-F1 points there (Zero-Shot Clinical Trial Patient Matching with LLMs), but no reported absolute F1 was found on Lacuna for that system. The second is the 51-profile synthetic oncology trial set against the ClinicalTrials.gov registry. We will also re-create cases in the format of the 20 simulated gastrointestinal cases to compare against the strongest baseline.

The metrics are claim-level correctness by claim type, recommendation completeness, cascade rate under injected upstream errors, evidence accuracy, the size of the human-review queue and clinician minutes per case, and feasibility-adjusted actionability. Feasibility-adjusted actionability is the fraction of a system's recommendations that are both correct and judged deliverable within the patient's documented window. Retrospective agreement with what was actually delivered is descriptive only. We will not make survival claims.

The decisive checks are three. At matched tokens, our system must show a lower cascade rate than critic debate. Replacing deterministic checkers with model critics must erase that advantage. And the single orchestrator given the same checkers as tools must fall short, or else we conclude that verification, not decomposition, is what matters. Any of these results is informative, and the third would change what the field should build.