Spending Verification Compute Where It Pays: Budgeted Policies for Multi-Pass Clinical Extraction

Large language models can pull structured facts, such as diagnosis codes, medications and trial arms, out of free-text clinical notes. They do this better when they check their own work. In the self-verification recipe of Self-Verification Improves Few-Shot Clinical Information Extraction, a first extraction is followed by three more passes. The first looks for missed items, the second finds a supporting quote for each item, and the third prunes items that have no support. This raises F1 on MIMIC-IV ICD-10 coding from 0.487 to 0.533, and raises GPT-4 F1 on EBM-NLP trial-arm extraction from 0.419 to 0.530. The cost is roughly four model calls per note, with long notes re-read several times. That is the latency problem the research file identifies as the main obstacle to deploying such pipelines in real-time workflows. Every note gets the same treatment, whether it is a two-line triage entry the first pass already handled perfectly or a sprawling discharge summary where most of the errors live.

This proposal treats verification passes as a scarce resource to be allocated, the way cluster schedulers allocate CPUs and GPUs. A small learned policy watches each note's first extraction and the current workload. It then decides, pass by pass, whether to stop, run the next verification step, or escalate to a larger model, all under a token or deadline budget. The bet is that most of self-verification's accuracy gain is concentrated in a minority of notes, and that cheap signals can identify that minority in advance. Self-verification could then be run at a fraction of its cost, with spending shifted toward hard notes when the system is quiet and away from them when it is busy.

Core Claim

We claim that a budget-aware allocation policy can recover most of self-verification's F1 gain on MIMIC-IV ICD-10 and EBM-NLP while spending a fraction of its model-call budget. We further claim that conditioning on queue load lets the policy meet latency deadlines under bursty arrivals where fixed pipelines fail. The resource-allocation literature supplies the method and the clinical-extraction literature supplies the workload. The direction reverses what the file's "unexplored intersection" sketch suggests. That sketch imagines reinforcement-learning cluster schedulers placing clinical NLP jobs on GPUs. We think that version mostly reduces to standard autoscaling of a model server, already handled by systems like KIS-S, and the clinical content would play no role in it. Here the clinical content is the decision variable: what is allocated is reasoning effort per document, and the payoff is measured in extraction accuracy.

The closest prior work makes adaptive decisions inside extraction but ignores cost. Diagnostic Inferencing via Improving Clinical Concept Extraction with Deep Reinforcement Learning trains a Q-learning agent to accept or reject candidate concepts against external evidence, reaching Recall@10 of 90% on 30 TREC CDS narratives against 60% for a non-RL baseline. Its actions edit the answer, not the compute spent, and it never saw long EHR notes. The count-reconciliation pipeline in An Agentic LLM Pipeline for Transforming Unstructured Clinical AI Literature into Structured Data gates extra passes with a hand-written trigger: a count mismatch launches up to two correction loops. That is a fixed rule of the kind we aim to replace with a learned one, and the authors themselves note that it checks structure rather than meaning. Self-verification itself supplies the action set, applied uniformly to every note.

Research Program

The environment is built once and then replayed. For every training note we run the full set of options and cache the result of each. Those options are the first extraction, then omission detection, evidence grounding and pruning in every valid order prefix, plus an escalation to a larger model. Because every counterfactual outcome is stored, the policy can be trained with exact offline rollouts. This removes the safety and sim-to-real worries that drove the construction of KISim in KIS-S. Building the cache costs about one full self-verification run over the training set; after that, training costs essentially nothing.

The state for a note at a given step contains six kinds of signal: note length and section structure; the number of items extracted so far and how much that count changed in the last pass; the model's token-level confidence on its extracted spans; the fraction of items that already have grounded evidence; the current queue length and arrival rate; and the remaining budget. The available actions are to stop, run the next verification pass, or escalate.

One constraint must be learned rather than assumed. The omission pass raises recall at the cost of precision, and it is the later grounding and pruning passes that restore precision. Stopping right after omission can therefore make results worse. A threshold rule cannot represent this, but a sequential policy can.

The objective trades extraction quality against cost:

$$J(\pi) = \mathbb{E}\left[\sum_i F_1(y_i^{\pi}, y_i^{*}) - \lambda \sum_i c_i^{\pi}\right] \quad \text{subject to the deadline-miss rate} \le \delta,$$

where $y_i^{\pi}$ is the policy's final extraction for note $i$, $y_i^{*}$ the gold labels, and $c_i^{\pi}$ the tokens spent on note $i$. The multiplier $\lambda$ sets the cost-quality trade-off, and $\delta$ caps the fraction of notes finished after their deadline. Sweeping $\lambda$ traces a Pareto curve of quality against cost, analogous to the waste-versus-underallocation frontier reported in Optimizing Memory Allocation in Distributed Clusters with Predictive Modeling.

We train with REINFORCE, as in DeepPlace, and with PPO, as in KIS-S. A Lagrangian update adjusts the weight on the deadline constraint during training. For the load-aware setting, notes arrive according to the Ramp, Periodic, Random and Spike patterns used to evaluate KIS-S, and model service time is fitted to measured per-call latency.

Baselines

The naive reference points are the two endpoints. The first runs the first extraction only. The second runs full four-step self-verification on every note, exactly as specified by Gero et al. Both use the same base model, prompts and data split, so that our reproduction can be checked against their reported 0.487 and 0.533 F1 on MIMIC-IV ICD-10 and 0.419 and 0.530 on EBM-NLP. Any method we propose must lie on or above the line joining these two points to be worth anything.

The strongest existing competitor is a confidence-gated rule in the spirit of Bayarbaatar's count-mismatch trigger. It runs the full verification chain only when the first extraction's mean span confidence falls below a threshold, or when a cheap re-count disagrees with the item count. We tune the threshold on the same validation set, with the same number of configurations as the learned policy, and evaluate at matched token budgets. Beating it means higher F1 at equal tokens across most of the budget range, not at a single cherry-picked point.

The second strong competitor predicts gain and then allocates greedily. A gradient-boosted regressor, trained on the cached outcomes, predicts the F1 gain of full verification for each note, with a calibrated safety margin following the quantile-regression design of Bader et al. Notes are then verified in order of predicted gain per token until the budget runs out. This baseline isolates the key ingredient of the proposal. If greedy allocation on predicted gains matches the sequential policy, then what matters is the value-of-compute predictor and not reinforcement learning. We would report that result as such.

A final ablation removes the queue features from the policy's state. It isolates whether load awareness, rather than per-note difficulty, accounts for the gains in the bursty-arrival experiments.

Why not just…?

Why not just run full self-verification and buy more GPUs? Under offline batch coding this is a fair answer, and we say so: if budgets are not binding, the full pipeline is the right choice. The proposal matters only where cost or deadlines bind, such as triage-time alerting or hospital-scale coding backlogs. The sepsis work of Horng et al. shows the value of reading notes at the moment of triage, raising AUC from 0.67 with vitals alone to 0.86 with notes added. Our bursty-arrival experiments measure directly how often the uniform pipeline misses deadlines at fixed capacity.

Why not just gate on model confidence? This is the real threat, and it is the comparison we run first. Two pieces of evidence suggest it may fall short. Confidence scores from instruction-tuned models are often poorly calibrated on rare codes, and the long tail is exactly where MIMIC-IV-ICD10 is hardest: macro F1 is 5.42% despite a macro AUC of 97.07% in Mimic-IV-ICD. A single threshold also cannot express that omission without pruning hurts precision. Still, the resource-allocation literature carries a warning. In Leveraging Reinforcement Learning for Task Resource Allocation in Scientific Workflows, a simple feedback loop wasted 601 GBh of memory against 2,060 GBh for gradient bandits and 13,418 GBh for Q-learning. Simple rules can win. If the tuned gate is within noise of the policy, we will recommend the gate.

Why not just use a smaller fine-tuned extractor and skip verification? Fine-tuned models such as DRG-LLaMA and CheXbert are fast and accurate for fixed label sets. CheXbert labeled 190,000 reports in about 18 minutes on one GPU. But such models need task-specific labeled data and give no evidence spans. Self-verification works few-shot and produced quotes with 93% overlap with expert annotations. We treat a fine-tuned PLM-ICD-style coder as an extra reference point on MIMIC-IV ICD-10 and include "escalate" or "defer to the cheap model" as policy actions, so the comparison is settled within one cost-quality plot.

Proposed Experiments

The main experiment draws Pareto curves of F1 against tokens per note on MIMIC-IV ICD-10, following the Self-Verification setup, and on EBM-NLP. CASI medication status serves as a third task; for CASI, no reported result was found on Lacuna. The decisive check is whether the learned policy recovers at least 80% of self-verification's F1 gain while using at most 40% of its extra tokens, and whether it beats both the confidence gate and greedy gain allocation at matched budgets. Statistical significance is assessed with bootstrap resampling over notes.

A second experiment replays the four arrival patterns at a fixed server capacity. It reports deadline-miss rate and F1 for the uniform pipeline, the gate, greedy allocation, and the policy with and without queue features. Load awareness is supported only if the policy with queue features cuts deadline misses under Spike traffic while giving up less F1 than the version without them.

A third analysis checks where the budget goes, split by note length, code frequency and pass type. It tests whether the policy learns the omission-then-prune dependency, and whether verification on rare codes is protected or starved. The second outcome would be a clinically important failure mode, and we would report it prominently.

Finally, we test transfer. A policy trained on ICD-10 coding is applied to EBM-NLP without retraining, to see whether value-of-compute signals carry across extraction tasks or must be relearned for each one.