Relax Retrieval, Then Check Every Constraint: Verdict-Driven Query Relaxation for Clinical Search

Precision-medicine search queries stack several hard conditions: a cancer type, a specific gene variant such as BRAF V600E, an age, a sex. Strict search engines often return nothing, because the corpus lacks any document that meets every condition at once. The standard fix is query relaxation, which means dropping or softening conditions one at a time until enough documents come back. That fix creates a second problem. Each relaxed query brings in near-misses, such as documents about the right cancer but the wrong mutation, or the right mutation in the wrong population. A downstream language model then treats these as relevant, which is exactly the distraction failure documented for readers flooded with irrelevant passages. Today the relaxation loop and the reader are designed separately. The loop stops when enough documents are returned, not when enough documents that actually satisfy the user's conditions are found.

We propose to close that loop with a small, domain-tuned model that reads each retrieved candidate and fills in a fixed, machine-checkable record with one verdict per original query condition: satisfied, violated, or not stated. Grammar-constrained decoding guarantees the format. The relaxation loop then stops based on how many candidates are verified to meet the required conditions, and ranking uses those per-condition verdicts. The claim is that forcing the reader into a per-condition verdict schema, with an explicit "not stated" option, turns the noise that relaxation creates into a verification task a compact model can do reliably. We also claim this beats both count-based relaxation and generic relevance reranking on the zero-result-prone queries where relaxation matters most.

Core Claim

The unit that relaxation manipulates is a constraint, yet every system in the file judges its output at the level of whole documents or result counts. The claim has three parts. A reader that emits constraint-level verdicts, rather than a single relevance score, recovers the precision that relaxation gives up. Using those verdicts as the stopping signal for relaxation reaches a better precision–recall point than any fixed result-count threshold. A roughly 8B model specialized on verdicts derived from registry metadata can do the verification as well as a prompted frontier model, at a small fraction of the cost.

Stated as a rule, let $C$ be the original query's conditions, $R \subseteq C$ the ones the user marks or the topic implies as required, and $v_c(d) \in \{\text{sat}, \text{viol}, \text{unk}\}$ the reader's verdict on condition $c$ for document $d$. At relaxation level $\ell$, retrieval returns a pool $P_\ell$. The loop stops at the first $\ell$ for which

$$|\{d \in P_\ell : v_c(d) = \text{sat} \;\; \forall c \in R\}| \ge k,$$

and ranks the union of pools by the weighted count of satisfied conditions minus violated ones. Unknown verdicts are neutral, which lets the reader abstain in the way that DAS does at the document level.

How This Relates to Prior Work

The relaxation mechanics come from existing systems. ECNU-ICA at TREC 2019 Precision Medicine rewrote Elasticsearch queries by downgrading strict Must clauses to soft Should clauses when too few documents matched. It reranked with a domain doc2vec model. We reuse its relaxation operators unchanged and replace both its count-based trigger and its vector reranker. MDKB-Bot supplies the split between required and optional conditions, which in that work was a static rule for querying a dialogue database. RELAX shows that symbolic relaxation adds 30–50% relative MRR on top of neural query answering over knowledge graphs. That result motivates keeping relaxation explicit rather than hoping dense retrieval absorbs it. However, RELAX scores answers by counting graph paths, not by checking text.

The noise literature motivates the verification step and constrains its design. DAS showed that letting a small zero-shot reader answer "unanswerable" on irrelevant documents raises exact match by 64% on average. Our "not stated" verdict extends that abstention to each individual condition. CompAct stops reading documents once its compressor judges the evidence complete. This is the closest analogue to our stopping rule, but CompAct stops consuming a fixed retrieval list, whereas we decide whether to change the query. Ext2Gen and SKR make readers robust to a fixed noisy pool or decide whether to retrieve at all. Neither reformulates a failed query.

Two other lines supply the method. On structured output, SynCode masks invalid tokens during decoding and removes all JSON syntax errors on JSON-Mode-Eval with models as small as Gemma2-2B. CodeIE shows that free-text serialization leaves 10–30% of outputs unparseable, while code-style templates bring this near zero. Both were evaluated on clean inputs. Whether constrained decoding stays reliable, or produces confident but wrong "satisfied" verdicts, when inputs are relaxation-induced near-misses is untested. We measure this directly. On specialization, EC-RAFT showed that a LoRA-tuned Llama-3.1-8B beats zero-shot Gemini-1.5-Flash on clinical trial criteria generation. It used single-shot retrieval and no enforced schema. Specializing Smaller Language Models towards Multi-Step Reasoning warns that narrow tuning collapses general reasoning. This matters less here because the verifier has only one job, but it means we must test transfer to condition types not seen in training.

Research Program

We would build three components.

The first is a relaxation engine over PubMed abstracts and ClinicalTrials.gov using BM25 with the ECNU-ICA operators. It produces a sequence of pools $P_0, P_1, \dots$ and records which conditions were softened at each step.

The second is the verifier: Llama-3.1-8B-Instruct with LoRA, matching the EC-RAFT and Ext2Gen setups. Its output is a fixed JSON record with one field per condition type (disease, gene or variant, age, sex, other) and a short evidence span per field. Output is enforced with SynCode-style grammar masking.

Training labels come without a teacher model. ClinicalTrials.gov records carry structured conditions, age limits, sex eligibility, and keyword fields, and the EC-RAFT corpus holds 267,347 such trials. We construct queries that match or deliberately mismatch a given trial on chosen conditions, and the registry fields provide the true verdicts. When a condition is absent from both the metadata and the text, the label is "not stated." For abstracts, we use gene and disease mentions only where the label is unambiguous. This avoids the teacher dependence the file flags for distillation pipelines. The cost is that registry-derived labels may not cover the phrasing found in abstracts, which the transfer tests below address.

The third component is an analysis study run across relaxation levels. It measures format validity and verdict accuracy with and without grammar masking as the share of near-misses in the pool rises. This directly probes the open question of whether masking protects against noise or amplifies it.

Baselines

The naive reference is the strict, unrelaxed Boolean query with BM25 ranking. On zero-result topics it returns nothing, and it sets the floor for recall.

The strongest existing system in the file on this benchmark is ECNU-ICA's heuristic Must/Should relaxation with domain query expansion and a doc2vec reranker. On TREC 2019 Precision Medicine it reports infNDCG 0.4672 and P@10 0.5675 on scientific abstracts (run sa_ft_rr), and infNDCG 0.5355 on clinical trials (run cl_ft_agg), per ECNU-ICA team at TREC 2019 Precision Medicine Track. The file gives no track-best numbers, so we treat these as the reference rather than as the ceiling. We will reimplement the pipeline on the same indexes and topics and confirm that we reproduce these numbers within noise before comparing. Beating it means higher P@10 and infNDCG on both subtasks. It also means a larger gain on the over-specified topic variants described below, where relaxation depth is greatest.

The competitor most likely to erase our gain is the same relaxation loop followed by an equally tuned 8B pointwise reranker. This reranker outputs one relevance score per document, is trained on the same registry-derived data collapsed to "all required conditions satisfied or not," and runs under the same LoRA budget, token budget, and hyperparameter search. A second fair competitor is dense retrieval to top-$k$ with no relaxation, read by an Ext2Gen-style DPO-tuned 8B model. Ext2Gen reports an overall score of 0.557 against a 0.473 default on its own noisy-chunk benchmark. No reported result was found on Lacuna for Ext2Gen on TREC Precision Medicine.

The ablations isolate each ingredient. We replace verdict-based stopping with the count threshold while keeping the verdict ranking. We collapse per-condition verdicts to a single score. We remove the "not stated" option. We turn grammar masking off, with and without post-hoc JSON repair. We give the verifier only the relaxed conditions instead of all original conditions.

Why not just…?

Why not just relax by count and rerank with a strong LLM reranker? This is the real threat. A good reranker trained on the same data may already learn to penalize wrong-mutation documents. Nothing in the file shows it cannot. The collapsed-score baseline above tests exactly this, with identical model, data, and compute. If per-condition verdicts do not beat it on P@10 for the over-specified variants, the schema is not the contribution. The proposal then reduces to verdict-based stopping, which we test separately.

Why not just use dense retrieval, which never returns zero results? Dense search always returns something, but on over-specified queries that something is the nearest near-miss. The system also gives no signal that it has silently dropped the gene constraint. ECNU-ICA reports that general neural embeddings did not beat heuristic rewriting plus domain vectors on this track. RELAX's gains over neural query answering point the same way in knowledge graphs. The dense-plus-Ext2Gen baseline tests this on our data.

Why not just prompt a frontier model with the whole relaxed pool? EC-RAFT's 8B model beat zero-shot Gemini-1.5-Flash on BERTScore (86.23 vs. 82.18) and clinical relevance (1.715 vs. 1.34). DAS and CompAct document accuracy loss and cost growth as context grows. Still, verification is a narrow task that a frontier model might do well. We will run a frontier model with the identical schema on a subset of topics and report both accuracy and cost per topic. A tie on accuracy would still leave the cost argument, and we will say so.

Why not just generate freely and repair the JSON afterward? For an eight-field verdict record, syntax repair may well suffice. The more important question is whether masking changes the verdicts themselves under noise. The masking on/off ablation, stratified by relaxation level, answers that. If masking brings no gain, we will drop it.

Proposed Experiments

The primary evaluation uses the 40 TREC 2019 Precision Medicine topics on both the abstracts and clinical-trials subtasks, measured with infNDCG and P@10.

To stress the zero-result regime, we add over-specified variants of each topic. These append realistic extra conditions, such as a second variant, a stricter age band, or a prior therapy, that we verify produce empty strict results. The original relevance judgments are then kept only for documents that still satisfy the added conditions according to registry fields. This makes the variants partly silver-labeled, and we report them separately.

Verdict accuracy per condition type is measured on held-out registry-derived trials. It is further split by relaxation depth to expose the noise curve. We track how often the model says "satisfied" when the truth is violated or not stated, since that is the error that matters clinically.

To test the specialization cost, we hold out one condition type (for example, sex) from training and evaluate on it. We also evaluate on abstracts only, which checks transfer from registry-style text to scientific prose.

The decisive checks come down to three questions. The first is whether per-condition verdicts beat the equally trained single-score reranker on P@10 for the over-specified variants. The second is whether verdict-based stopping beats the best count threshold, chosen on validation, at matched average relaxation depth. The third is whether grammar masking changes the verdict error rate at deep relaxation levels, separately from its effect on syntax. A yes to the first two supports the core claim. A no to the first means the contribution is the stopping rule alone, and we would report it that way.