Checking Trial Eligibility as Time-Aware Queries over Patient Timelines

Most clinical trials struggle to enroll enough patients, and a large share fail outright for that reason. The survey on LLM-assisted recruitment reports that about 80% miss enrollment deadlines. Language-model matchers now screen patients cheaply: about $1.55 per patient against an estimated $34.75 for manual screening in Zero-Shot Clinical Trial Patient Matching with LLMs. Their characteristic error is about time. They exclude a patient for "prior chemotherapy" that was given years ago for a condition later found to be misdiagnosed, which TrialGPT 2.0 calls the "Literalness Trap". They also confuse a problem resolved a decade ago with an active one, which the survey calls "information isolation". Eligibility criteria are often statements about time: "within 6 months," "progression after first-line therapy," "stable for at least 4 weeks." Yet matchers read either single notes or condensed vignettes.

We propose to treat eligibility as a query about when things happened and whether they are still true. A language model reads each note once and records dated clinical facts. Each fact is tagged as active, resolved, or refuted, and a fact can be marked as superseding an earlier one. Separately, each criterion is translated into a logical formula whose terms carry time windows and change conditions. A small deterministic program then evaluates the formula against the timeline as of the screening date. In plain terms, the model reads, and a program handles dates and logic, which models do badly. The claim to be tested is specific: this split should sharply cut false exclusions and window-boundary errors on multi-note records, compared with prompting a strong model over the whole record, at lower cost and with an audit trail.

What exists and where this differs

The closest design is the "structure-then-match" pipeline of Scaling Clinical Trial Matching Using Large Language Models. It has GPT-4 rewrite criteria as an OR of AND-clauses (disjunctive normal form), extracts patient attributes with biomedical models, and executes the logic deterministically. We reuse that method. Its patient attributes are static, though, and the authors name longitudinal record synthesis as their main unresolved bottleneck. What we add is time stamps, fact status, supersession links, and temporal operators in the criterion language.

Two neural systems also match criteria to longitudinal records: DeepEnroll, which pools hierarchical visit embeddings, and COMPOSE, which lets each criterion attend over a memory of medical codes. Both read only structured billing codes. Neither represents whether a condition has resolved or when a window closes. COMPOSE's finding that organized temporal memory holds accuracy as histories lengthen motivates our design; we move that idea from codes to narrative notes.

The LLM matchers supply the strongest baselines. TrialGPT and TrialGPT 2.0 score criteria with chain-of-thought prompting over short vignettes. Zero-Shot Clinical Trial Patient Matching with LLMs evaluates criteria against each note separately (its ACIN strategy) or against retrieved note chunks. Neither ever combines evidence across notes in time.

ChatEHR constrains the design. It ingests whole multi-year timelines, yet averages 0.73 hallucinations and 1.60 inaccuracies per generated summary. That result argues against making a free-text summary the intermediate step.

From longitudinal imaging we borrow a principle rather than an architecture. DeltaNet updates a prior report through a gate that drops findings that are no longer true. BioViL-T separates static from change features and improves report factuality (CheXbert score from 29.3 to 35.86). Both show that representing change explicitly beats judging a single snapshot. Our supersession links are the text analogue of DeltaNet's gate.

Core Claim

The claim is that temporal errors in matching come mostly from the representation and not from missing model capacity. A patient timeline of dated, status-tagged facts, combined with criteria compiled into time-aware logic, should remove most of these errors. It should also let a small locally hosted extractor match a frontier model reading the full record. The executor's rule is simple. A criterion term with concept $c$, window $w$, and screening date $t_0$ is satisfied when

$$\exists f:\ \mathrm{concept}(f)=c,\ \mathrm{status}_{t_0}(f)=\text{active},\ t_0 - t_f \le w,$$

where $f$ ranges over extracted facts and $t_f$ is the fact's date. Here $\mathrm{status}_{t_0}(f)$ is the fact's status after applying every supersession recorded before $t_0$. Change conditions such as "progressed after X" become ordering constraints between two facts. Any term that cannot be evaluated returns "missing information" instead of a guess, mirroring TrialGPT 2.0's output categories.

Research Program

We will build three components. The first is a per-note fact extractor, a prompted open model small enough to run inside a hospital. For each note it outputs concept, value with unit, date (explicit or resolved relative to the note date), status, and optional pointers to earlier facts it corrects or resolves. The second is a criterion compiler. It extends the disjunctive-normal-form parser of the oncology case study with window, ordering, duration, and change operators. The third is a roughly few-hundred-line executor that returns a decision and the exact facts used.

Error analysis will attribute each mistake to extraction, compilation, or execution. That split is itself a result the field lacks, because the survey's failure taxonomy has not been measured by pipeline stage.

Baselines

The naive reference applies the same prompted model to only the most recent note. It measures how much any use of history matters.

The strongest existing method is zero-shot GPT-4 with the ACIN strategy from Zero-Shot Clinical Trial Patient Matching with LLMs, together with its two-stage retrieve-then-score variant. It is the best reported system on n2c2 2018, where it beat the prior best by +6 macro-F1 and +2 micro-F1. The file gives no absolute GPT-4 score. The survey reports that top n2c2 systems reach 0.91 micro-F1, which we take as the level to match. For fairness, every system gets the same records, the same criterion text, the same model family for a strong-model condition (both our extractor and the baseline use the same frontier model), and the same small-model condition. Prompt tuning is capped at equal budgets on a shared development split.

We also run long-context prompting, with the whole record and the criteria in one prompt using chain of thought, and TrialGPT-Matching applied to an LLM-written summary of the record. The summary baseline tests whether summarizing first is enough.

Beating the baselines means matching the 0.91 micro-F1 level on unmodified n2c2 and cutting error substantially on the temporal test suite described below. We pre-register a halving of false exclusions on resolved or refuted perturbations and a halving of window-boundary errors relative to the best baseline. We also report cost per patient against the $1.55 ACIN figure.

Three ablations isolate the key ingredient. The first removes status and supersession, leaving dated facts only. The second removes dates, which reduces the system to the static structure-then-match pipeline. The third keeps the timeline but replaces the executor with an LLM reading the timeline. The third separates whether the representation or the deterministic execution does the work.

Why not just…?

Why not just give a long-context frontier model the whole record? This is the real threat. On n2c2's short records (2 to 5 notes, about 2,700 words per patient) it may do as well as our system. We therefore do not rest the claim on n2c2 alone. The temporal test suite and the record-length curve decide it. If long-context prompting matches us there, our remaining contribution is cost, local deployment, and auditable evidence, and we will say so. The prior evidence that it will fall short is indirect: ChatEHR's error rates on long timelines, the survey's report of about 15x latency for frontier models, and the survey's finding that information isolation dominates failures.

Why not just retrieve the relevant chunks and prompt, as the two-stage pipeline does? Retrieval chooses text by similarity to the criterion. The note that says "the chemotherapy was for a misdiagnosis" may share little vocabulary with "prior chemotherapy," so the correcting evidence goes unretrieved. Our supersession perturbations are built to test exactly this case. Retrieval quality will be measured directly.

Why not just summarize the timeline and run TrialGPT on the summary? A summary is lossy and prone to hallucination, as ChatEHR's inaccuracy rates show, and it removes exact dates, which window criteria require. The summary baseline runs as a matter of course. If it is competitive on window-boundary cases, the dated-fact representation is unnecessary.

Why not just write rules? Hand-written rules reached 93.6% recall in the oncology case study but took over 450 expert hours. They also cannot read narrative corrections. Our compiler generates such rules automatically, so the comparison is authoring cost at equal recall.

Proposed Experiments

The first experiment is standard n2c2 2018 criterion-level macro-F1 and micro-F1. We split criteria by whether they carry a time window or a history-dependent condition, and report each group separately.

The second is the main test: a temporal counterfactual suite built from n2c2 records. We insert a later note that resolves or refutes an earlier finding. We shift note dates so a qualifying event falls just inside or just outside a criterion window. We add distractor notes that restate old problems in the present tense. Each edit changes the gold label in a known way, so the suite measures false exclusions, false inclusions, and boundary accuracy with no new annotation. A clinician will check a sample of edits for plausibility. The labels are only as reliable as n2c2's own annotation, which has an inter-annotator kappa of 0.54, so we will report results on the high-agreement subset separately.

The third experiment varies record length. We concatenate the same patient's notes with realistic filler encounters and plot accuracy as length grows, testing COMPOSE's finding for narrative text.

The fourth is a small prospective-style study at a partner site on real multi-year oncology records. It uses TrialGPT 2.0's reason categories and clinician screening time as outcomes, with the reported drop from 129.8 to 58.4 seconds per patient as the reference.

The decisive check is simple. If the ablation without status and supersession loses most of the gain on the resolved and refuted perturbations, and the LLM-executor ablation loses the gain on window boundaries, the two design choices are each justified. If neither ablation loses much, the idea is wrong and a well-prompted strong model is sufficient.