TrialJev: Open-Weight Single Pass Eligibility Scoring for Patient Trial Screening

Omar Ibrahim, Grant Hamblin, Benjamin Haibe-KainsarXiv 2026 · arXiv

TL;DR

TrialJev is an open-weight model designed for rapid patient-trial eligibility screening. By using a pointer head to score natural-language options in a single forward pass, it achieves accuracy comparable to generative LLMs while being nearly 3x faster and running entirely on local hardware, preserving patient privacy.

What Problem Does This Paper Address?

Matching patients to clinical trials is a high-stakes needle-in-a-haystack problem. A single patient might be a candidate for hundreds of trials, each with complex, nested eligibility criteria involving numeric thresholds, temporal constraints, and medical negations.

While Large Language Models (LLMs) have shown promise in this area, they face two major hurdles:

  1. Latency: Generative LLMs are slow because they decode verdicts token-by-token (autoregressive decoding). When screening thousands of patient-trial pairs, this cost becomes prohibitive.
  2. Privacy: Many high-performing models are behind proprietary APIs. Sending sensitive patient records to external servers often violates hospital data governance and privacy regulations.

The authors introduce TrialJev to bridge this gap, providing a fast, local, and accurate "first-pass" scorer to prioritize candidates for deeper clinical review.

Methods

TrialJev adapts the "typed-decision" architecture (previously seen in MedJev and kev) to the task of clinical trial eligibility.

  • Architecture: It uses Qwen backbones (ranging from 0.8B to 27B parameters) adapted via Low-Rank Adaptation (LoRA).
  • The Single-Pass Trick: Instead of generating text like "Eligible" or "Excluded," TrialJev uses a bilinear pointer head. This head looks at the model's hidden states at a specific "decision" token and at the boundaries of the answer options. It then calculates a normalized score for each option in a single forward pass.
  • Inference Routes: The paper compares three distinct ways to get a verdict, as illustrated in Figure 1:
  • (a) TrialJev: Scores option spans directly from hidden states.
  • (b) A/B-logit Control: A classifier that looks only at the logits of tokens "A" and "B."
    • (c) Generative LLM: The standard route of prefilling, decoding, and parsing a text verdict.
Scoring and writing are distinct inference routes. (a) TrialJev scores option spans from hidden states in one pass. (b) The A/B-logit control scores two label tokens. (c) Generation prefills, decodes and parses a verdict.

Datasets

The models were trained and evaluated using the TREC Clinical Trials benchmarks:

  • Training/Development: TREC 2021 data, consisting of 75 synthetic patient cases and approximately 35,000 judgments.
  • Evaluation: TREC 2022 data, featuring 50 unseen synthetic patients and 6,975 "eligible-or-excluded" pairs.
  • Task Definition: The task is binary classification: determining if a patient is eligible or excluded for a trial they have already been deemed "relevant" to.

Key Insights and Ideas

  • Decoding is the Bottleneck: The primary efficiency gain comes from avoiding autoregressive decoding. An ablation study (the A/B-logit control) showed that simply not writing text accounts for most of the speedup, rather than the specific pointer head architecture itself.
  • Local Models Match Proprietary APIs: With proper calibration (fitting a temperature and class-prior weight on development data), the open-weight 27B TrialJev models achieved accuracy statistically indistinguishable from the proprietary, hosted Jev API.
  • Initialization is Critical for Small Models: The smallest model (0.8B) failed to learn the task effectively from scratch. However, when initialized with weights from a related clinical extraction task (MedJev), its performance jumped by over 16 points.
  • Calibration Nuances: While TrialJev is fast and accurate, the pointer head tends to be under-confident compared to the A/B-logit control. This highlights that a normalized score is not automatically a calibrated clinical probability and requires post-hoc adjustment.
Reliability of TrialJev and the A/B-logit control. The pointer head (TrialJev) is mostly under-confident, while the logit control stays closer to the diagonal.

Key Results

  • Accuracy: The 27B models achieved 80.97% (Qwen3.5) and 81.84% (Qwen3.8) accuracy, outperforming generative LLMs fine-tuned on the same data (~79.9%).
  • Speedup: In matched batch-1 measurements, TrialJev was 2.45x to 2.70x faster than the generative approach at the 9B–27B scale. At 27B, it required only ~178 ms per patient-trial pair.
  • Latency vs. Accuracy: Figure 3 demonstrates the clear Pareto advantage of the single-pass approach, maintaining high accuracy while significantly reducing latency compared to generative and explanation-based baselines.
Scatter panels comparing accuracy and median latency. TrialJev (blue) maintains high accuracy while being significantly faster than fine-tuned generative LLMs (orange) and zero-shot explanation baselines (red).

Limitations and Future Work

  • Synthetic Data: The evaluation relied on synthetic TREC patients. Real-world clinical records are noisier and longer, which may impact performance.
  • Binary Scope: The current model only distinguishes between "eligible" and "excluded" for relevant trials; it does not handle the initial retrieval of relevant trials from a registry.
  • Lack of Explainability: Unlike generative models that can provide a rationale, TrialJev only provides a score. The authors suggest a staged workflow where TrialJev prioritizes candidates, and a slower, explanatory model reviews only the top-ranked pairs.
  • Future Directions: The authors note that the pointer architecture could be extended to handle multiple questions (e.g., checking individual criteria) over a shared patient state, which could lead to even greater efficiency gains for long patient records.

Summary updated

Abstract

Clinical trial screening requires assessing large numbers of patient-trial pairs to prioritize candidates for detailed eligibility review, often under restrictions on sharing patient records. We introduce TrialJev, an open-weight eligibility scorer that uses LoRA-adapted Qwen models and a pointer head to score natural-language answer options in a single forward pass, without autoregressive decoding. Trained on TREC Clinical Trials 2021 and evaluated on 6,975 eligible-or-excluded pairs from 50 unseen TREC 2022 synthetic patients, TrialJev achieves 80.97% (Qwen3.5-27B) and 81.84% (Qwen3.8-27B) accuracy, compared with 79.87% and 79.93% for LLMs fine-tuned on the same data to generate verdicts. In matched batch-1 measurements, verdict generation takes 2.45-2.70 times as long as the one-pass inference of TrialJev at 9B-27B. Under a common development-fitted decision correction, the two 27B TrialJev models reach 80.99% and 81.88% accuracy, compared with 81.28% for the proprietary hosted Jev API; paired confidence intervals include zero. TrialJev runs locally on a single GPU, requiring approximately 178 ms per pair at 27B, without transmitting patient text to an external model API. An A/B-logit ablation shows that avoiding decoding, rather than the pointer head alone, accounts for the measured efficiency advantage. TrialJev is intended for first-pass prioritization of candidate trials, complementing rather than replacing explanatory models and clinical review. Evaluation is limited to synthetic patients and implementation-specific timing, and clinical use has not been validated. Adapters, pointer heads, and model cards are released on Hugging Face.
BibTeX
@misc{ibrahim2026trialjev,
  author = {Omar Ibrahim and Grant Hamblin and Benjamin Haibe-Kains},
  title = {{TrialJev: Open-Weight Single Pass Eligibility Scoring for Patient Trial Screening}},
  year = {2026},
  eprint = {2611.10145},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10145},
}
Figure