Closed commercial language models are being placed in front of clinicians, but outside researchers can only reach them through an API: no weights, no training data, no view of the retrieval step. The standard outside-in way to catch their factual errors is to cross-examine them. One method samples the model many times and checks whether it agrees with itself. Another asks a second, open "verifier" model the same question and flags disagreements. Both methods fail when the closed model is consistently wrong, meaning it repeats the same confident falsehood every time. The cross-check only helps if the verifier does not share that falsehood. This proposal argues that the most useful property of a domain-pre-trained verifier is not that it is more accurate. It is that it was trained on a different body of text, so it makes different mistakes from a web-trained frontier model, and different mistakes are what a cross-check needs.
The concrete claim is testable. Take open models of matched size and continue their pre-training on biomedical text, then use them as verifiers against proprietary models on adversarial medical hallucination benchmarks. They should lower the rate at which the verifier endorses the target's specific wrong answer. They should do this more than instruction-tuning the same base on medical data, and more than using a larger frontier model as the verifier. The program measures that "shared-error rate" directly, because it, rather than verifier accuracy, sets an upper bound on how many of a closed model's errors any cross-check can catch.
Core Claim
Cross-model checking catches a target error only when the verifier disagrees with it. Let $T$ be the closed target model and $V$ the verifier. Their answers to a question $q$ are $a_T(q)$ and $a_V(q)$. The recall of a disagreement-based detector on the target's errors is
$$ \text{Recall} = 1 - \Pr\big[a_V(q) \equiv a_T(q) \,\big|\, a_T(q)\ \text{wrong}\big], $$
where $\equiv$ means "semantically equivalent," as judged by an entailment model. Call the subtracted term the shared-error rate $s(V,T)$. Verifier accuracy enters this quantity only indirectly. A verifier can be quite accurate and still share almost all of the target's wrong answers if both learned the same misconceptions from the same web text. The hypothesis is that continued pre-training on domain text lowers $s(V,T)$ against frontier targets by more than its accuracy gain alone would predict. Instruction tuning with low-rank adapters (LoRA, a cheap method that trains small added matrices rather than all weights) changes output style but leaves the shared priors largely intact, so it should not lower $s(V,T)$ in the same way.
This differs from the closest work in an identifiable way. SAC3 introduced cross-question and cross-model checking and showed it beats self-consistency when models are consistently wrong. It used a general verifier (Guanaco-33b) and only general-knowledge tasks, and it never measured why some verifiers help more than others. BLADE pairs a small domain-pre-trained model with a black-box model, but it uses the small model to supply knowledge that boosts multiple-choice accuracy, not to audit the black box. It also notes that the small model hallucinates itself. AutoMedEval builds an open medical judge on the domain-adapted MedLLaMA-13B and shows it tracks physicians better than GPT-4 does. That is a capability-scoring setting, not adversarial hallucination detection, and it offers no account of why the domain judge wins. This proposal supplies that account and tests it with controlled ablations.
Motivation and Constraints from Prior Work
Three results motivate the focus on consistent errors. SAC3 found that naive self-consistency collapses to 29.6% accuracy on an unbalanced senator-search task because the model repeats the same wrong answer. Rethinking Hallucinations went further: common detectors (entropy, surprisal, self-check) separate stable answers from unstable ones, not true answers from false ones. That paper also reports about 60% prompt-level ambiguity for Llama2-13B-Chat on the medical exam benchmark Med-HALT. MHB shows that even the best closed model, Claude-4-Sonnet, hallucinates on 29.1% of adversarial clinical items, while the open Llama-4-Scout reaches 59.0%. Together these say that a general open verifier is weak in medicine, and that a frontier verifier may err in the same places as the frontier target.
Two results constrain the design. SelfCheckGPT is strong where it was tested: AUC-PR of 93.42 on WikiBio, measuring how well it ranks non-factual sentences. It must therefore be a real baseline, not a strawman, even though it was only tested on Wikipedia-style biographies. BioBART shows that pre-training on formal PubMed text transfers imperfectly to informal patient dialogue. The verifier corpus therefore needs a clinical-narrative component, and results should be reported separately for exam-style and dialogue-style items.
The domain pre-training literature supplies the method and a prior expectation. Pre-training on isolated domain text preserves domain-specific meanings that are "drowned" in mixed corpora, as shown in A Deep Fusion Model for Domain Adaptation in Phrase-based MT and MonoCoder. In code, pre-training predicts downstream performance better than the choice of fine-tuning method, as shown in A Comparative Study of PEFT Methods for Python Code Generation. None of these papers tested hallucination or error correlation, which is the gap here.
Research Program
Proprietary opacity sets the constraint: targets are reached only through the API. Hallucination is the phenomenon under study. Domain pre-training is the manipulated variable.
Verifier family. We build a controlled family of open verifiers from one general base model of about 8B parameters. The first is the base itself. The second is the base plus LoRA instruction tuning on medical QA and synthetic hallucination labels, in the style of OPDAI. The third is the base plus continued pre-training on PubMed abstracts, using text infilling and keeping sentence order, as BioBART found that order matters. The fourth adds clinical case narratives (PMC-Patients-style reports) to that pre-training corpus. Variants three and four then receive the same LoRA instruction tuning as variant two, so instruction data is held fixed and only pre-training differs. Continued pre-training is the expensive part. As a cheaper replication of the comparison, we also use the existing pair of MedLLaMA-13B and its general LLaMA-13B parent, the backbone of AutoMedEval.
Targets. The targets are the closed models evaluated in MHB: GPT-4, Claude-4-Sonnet and Gemini 2.5 Pro. Every call logs the model version string and date. Vendors change models silently, as the Spring-2024 snapshot problem in Hallucination-Free? illustrates. All target responses are therefore cached once and released, so that verifier comparisons run against a fixed set of outputs rather than a moving endpoint.
Detector. The detection procedure follows SAC3's question-perturbation scheme. Each item is rephrased into several semantically equivalent versions. Target and verifier both answer every version, and the hallucination score is the rate at which the two disagree across versions. We additionally compute $s(V,T)$ against expert labels, along with the verifier's own accuracy. That lets us split each verifier's detection gain into two parts: what comes from being right more often, and what comes from being wrong in different places from the target.
Baselines
The do-nothing reference treats every target answer as correct. Its detection AUROC is 0.5, and its hallucination rate equals the target's raw rate on MHB, which for the strongest target is the 29.1% reported for Claude-4-Sonnet in MHB.
Self-consistency is SelfCheckGPT with its LLM-prompting scorer. It samples the target $N$ times at nonzero temperature and scores agreement with the main response, and it is the strongest method that needs no second model. Its reported AUC-PR is 93.42 on WikiBio GPT-3. We found no reported result on Lacuna for SelfCheckGPT on MHB, Med-HALT or CMHE, so we will measure it ourselves. To be fair, it gets the same total API call budget per item that the cross-check methods spend.
The strongest existing black-box method is SAC3 with a general open verifier. Its best reported numbers are AUROC 87.0% on HotpotQA-halu and 77.2% on NQ-open-halu with Guanaco-33b. We found no reported result on Lacuna for SAC3 on any medical benchmark. We run it with the same perturbation count as our method, using both Guanaco-33b and a modern general open model of matched size as verifiers. Beating it means a domain-pre-trained verifier of equal or smaller size gives higher AUROC than these general verifiers on MHB and CMHE-HD, with non-overlapping bootstrap intervals.
The frontier-verifier baseline is SAC3 with a different proprietary model as the verifier, for example Claude checking GPT-4. It is the most likely to win on accuracy alone. Physician-correlation results in AutoMedEval (Spearman 0.6314 against GPT-4's 0.5128) suggest a domain model can beat a frontier judge, but that is a different task. Beating it means equal or higher AUROC and, specifically, lower $s(V,T)$.
The key ablation is variant two against variant three in the verifier family: identical instruction tuning, with and without continued pre-training. If they tie on $s(V,T)$, the proposal's central mechanism is wrong.
Why not just…?
Why not just use a frontier model as the verifier? This is the most serious threat. A frontier verifier will probably be more accurate than an 8B domain model, and accuracy alone could carry it. The shared-error measurement settles the question directly. If frontier verifiers have a lower $s(V,T)$ against frontier targets than the domain verifiers do, the thesis fails and we will report that. If they are more accurate but share more of the target's wrong answers, an ensemble of frontier and domain verifiers should beat either one alone, and we test that combination explicitly. A frontier verifier is also a moving, unauditable target, which matters for reproducible auditing. But that is a secondary argument and cannot rescue a loss on the numbers.
Why not just retrieve the relevant source and check entailment against it? When a single authoritative source exists, as with citation-grounded legal answers, retrieval plus an entailment check may well match or beat any parametric verifier. We therefore include it as a baseline on the 202-query Hallucination-Free? benchmark. The claim here is restricted to closed-book clinical claims, which make up most of MHB, where reasoning errors and procedural errors have no single passage to check against. Retrieval is also known to add prompt-sensitive randomness: Rethinking Hallucinations finds it shifts errors from systematic to random. Separately, entailment models miss small but decisive lexical shifts, a limitation TRAC inherits along with prediction sets averaging 20.4 answers at 90% coverage. We report results split by item type so a reader can see where retrieval wins.
Why not just LoRA-tune a general model to detect medical hallucinations? Supervised detector tuning can work well within its own distribution. PHANTOM raised Qwen2.5-7B from F1 0.708 to 0.933 on 10-K filings. Verifier variant two is exactly this alternative, run under matched data and compute. We train it on CMHE-HD and evaluate it on MHB, a cross-benchmark transfer, to test whether instruction tuning generalizes to unseen trap types. If variant two matches variant three there, cheap tuning suffices and the pre-training cost is not justified.
Why not just sample the target more times? More samples sharpen the estimate of how stable the target is, not whether it is correct. SAC3 and Rethinking Hallucinations both show that stable wrong answers pass this check. Matching the call budget across methods makes this directly visible in our results.
Proposed Experiments and Decisive Checks
The primary benchmark is MHB, using its expert rubrics as labels and reporting AUROC and AUC-PR by trap category. Med-HALT measures how the target's prompt-level ambiguity interacts with detection. CMHE-HD from Detection, Diagnosis, and Explanation tests Chinese-language transfer. The legal benchmark from Hallucination-Free? serves as a stress test where retrieval should be strong.
The first decisive check is the matched ablation. Continued pre-training (variant three against variant two) must lower $s(V,T)$ against all three closed targets, and replicate on the MedLLaMA-13B versus LLaMA-13B pair.
The second check is an accuracy-matched comparison. We select checkpoints of a general verifier and a domain verifier that reach equal medical QA accuracy, then compare their $s(V,T)$. If the domain verifier still shares fewer of the target's errors at equal accuracy, the effect is about decorrelation and not merely competence.
The third check is the frontier-plus-domain ensemble against frontier-only verification. This shows whether the domain verifier contributes signal that a stronger general model lacks.
We count the proposal as successful if all three checks favor the domain verifier on MHB. If only the accuracy-matched comparison fails, we conclude that domain pre-training helps simply by making the verifier more accurate. That is a weaker, still publishable claim, and we would report it as such.
References
- SAC³: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency
- BLADE: Enhancing Black-Box Large Language Models with Small Domain-Specific Models
- AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation
- Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity
- MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks
- SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
- BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model
- A Deep Fusion Model for Domain Adaptation in Phrase-based MT
- MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks
- A Comparative Study of PEFT Methods for Python Code Generation
- OPDAI at SemEval-2024 Task 6: Small LLMs can Accelerate Hallucination Detection with Weakly Supervised Data
- Hallucination‐Free? Assessing the Reliability of Leading <scp>AI</scp> Legal Research Tools
- TRAC: Trustworthy Retrieval Augmented Chatbot
- PHANTOM: A Benchmark for Hallucination Detection in Financial Long-Context QA
- Detection, Diagnosis, and Explanation: A Benchmark for Chinese Medical Hallucination Evaluation