Credibility-Aware Retrieval Without Generic Hedging

Retrieval-augmented generation (RAG) lets a language model look up documents before it answers. The trouble is that some of the documents it finds come from tabloids, press releases, or outright disinformation. A growing set of methods teaches the model to discount those documents. They filter them out, down-weight them, or annotate them with publisher track records. These methods are judged almost entirely on whether the final short answer is correct.

This proposal starts from a failure those evaluations cannot see. Suppose the specific fact a user asks for appears only in a low-credibility document, and the model is told to distrust that document. It then has two bad options. It can fall back on a vague, uninformative reply ("sources vary; it depends on several factors"), which is the classic generic response bias of likelihood-trained models. Or it can fill the gap from its own memory with a confident invention, which is a hallucination. The desirable behavior is a third option, which we call a specific abstention: "The two reputable outlets do not give a date; the only source stating March 2021 is a site with a poor factual record." We propose to measure all three behaviors together and then train models to prefer the third.

Core Claim

We claim three things. First, current credibility interventions convert confabulation into generic hedging rather than into informative answers. Second, one quantity separates a generic hedge from a specific abstention: how much the response depends on the retrieved evidence, measured as pointwise mutual information (PMI) between the response and the evidence. A hedge is about equally likely whatever documents were retrieved. A specific abstention names what the credible sources do and do not say, so it is highly evidence-dependent. Third, a preference-tuning reward can combine claim-level credibility-weighted grounding with this evidence-dependence term. Such a reward should push models toward answers that are as specific as the credible evidence allows, and no more specific.

The new element is the joint accounting and the evidence-dependence criterion for genericity. Neither credibility-aware generation nor hallucination-aware tuning currently measures it.

Where Prior Work Leaves the Gap

Three credibility methods motivate the diagnostic, and each reports only answer accuracy.

  • CrAM locates the attention heads that carry misinformation into the output and scales down attention to low-credibility tokens. It reports Exact Match only.
  • Resolving Conflicting Evidence in Automated Fact-Checking introduces the CONFACT benchmark. It shows that appending publisher background to the prompt (Source Background Augmentation, SBA) beats filtering or re-ranking at retrieval. Its outcome is a binary support/refute label.
  • RA-RAG estimates source reliability without supervision and aggregates answers by weighted voting. It evaluates on factoid QA only.

None of the three asks what the model says when credible evidence is incomplete. We reuse their interventions as treatments.

Two lines of work supply the method we build on.

  • RAGTruth provides span-level hallucination labels. It shows that models more often add unsupported details than contradict the source.
  • RAG-HAT turns detected hallucinations into preference pairs for Direct Preference Optimization (DPO), a method that fine-tunes a model directly on pairs of preferred and rejected outputs.

We borrow the detector and the DPO pipeline. RAG-HAT, however, assumes every retrieved document is true, and it guards only against shorter outputs through sentence deletion. It never penalizes a full-length but content-free hedge. That missing penalty is our contribution.

Three further papers constrain the design.

  • Chain-of-Specificity raises specificity by chaining constraints over the model's own knowledge. This is exactly the pressure that should produce confabulation when grounding is missing. We use it as the "force specificity" contrast.
  • TRAC guarantees coverage with conformal prediction sets, meaning sets of candidate answers built to contain the true answer at a chosen rate. At 90% coverage on Natural Questions it returns an average of 20.4 answers. That result shows how guarantees can collapse into set-level genericity.
  • The Unintended Trade-off of AI Alignment finds that truthfulness interventions can damage refusal behavior. We therefore must check that our tuning does not do the same.

Research Program

Controlled evidence conditions. For each question we control what the credible sources contain, under three conditions:

  • Full: credible documents contain the specific answer.
  • Partial: credible documents contain part of it, and a low-credibility document supplies the remainder, possibly falsely.
  • Absent: only low-credibility documents carry the specific detail.

The target behavior differs by condition: a specific answer, a specific answer with the contested part explicitly attributed, and a specific abstention.

We build these conditions from existing resources:

  • the TriviaQA "four real documents plus one fake" setup used by CrAM, adding variants in which real documents carrying the answer are removed;
  • CONFACT's human-verified conflict split (HumC), with its publisher metadata, recast from verdict prediction to a short explanatory answer;
  • the false-premise and long-tail-entity slices of CRAG, where specific credible evidence is naturally scarce.

Scoring each response. We decompose every response into atomic claims. We then separate asserted claims from attributed claims such as "a tabloid reports X." This separation lets a specific abstention mention a dubious claim without being charged for asserting it.

Each asserted claim is checked for entailment against credible and non-credible documents separately. The checker is a hallucination detector fine-tuned on RAGTruth, whose published response-level F1 is 78.7%. Each claim falls into one of three bins:

  • credible-supported: $N_c$, the number of asserted claims entailed by a credible document;
  • low-credibility-only: $N_\ell$, the number entailed only by low-credibility documents;
  • unsupported: $N_u$, the number entailed by no document.

Measuring evidence dependence. The genericity score is length-normalized PMI:

$$\mathrm{Dep}(y) = \frac{1}{|y|}\left[\log p_\theta(y \mid q, D) - \log p_\theta(y \mid q, \tilde D)\right]$$

Here $y$ is the response, $q$ the question, $D$ the retrieved documents, $\tilde D$ documents retrieved for an unrelated question, $|y|$ the response length in tokens, and $p_\theta$ a fixed scoring model. Generic responses score near zero because MLE training favors phrases that are plausible across many contexts, as described in the NTCIR-13 overview. Before relying on $\mathrm{Dep}$, we validate it against human genericity judgments on about 500 responses.

Combined reward. We use the reward

$$R(y) = N_c - \lambda N_\ell - \mu N_u + \gamma\, \mathrm{Dep}(y),$$

where $\lambda, \mu, \gamma \ge 0$ are weights. We will report the tradeoff across a range of weights rather than a single setting. To build preference pairs, we sample several responses per training instance and rank them by $R$. Using Llama-3-8B, Qwen, and Mistral-class models, we then run DPO following the RAG-HAT recipe.

Guarding against a gamed score. Verbatim copying of evidence could inflate $\mathrm{Dep}$. We therefore report an extractive-copy baseline, and we cap credit for claims restated word-for-word.

Credibility scores. Credibility is taken from benchmark metadata. Because this is an oracle, we repeat the evaluation with RA-RAG's estimated reliability scores to test robustness to noisy credibility.

Baselines

Vanilla RAG (the do-nothing reference). The untuned model with all documents concatenated. This fixes the starting rates of confabulation and hedging in each condition. As an anchor, CRAG reports straightforward RAG at about 44% accuracy and a truthfulness score of about 13.4% for GPT-4 Turbo.

CrAM, the strongest existing credibility method. CrAM is the strongest inference-time competitor, and it acts on exactly the signal we care about. We rerun its head-localization calibration on our models using the same calibration budget it prescribes. On TriviaQA with one fake document, CrAM reports gains over prompt-based credibility of +25.5% EM (Qwen), +31.9% (Llama2), and +10.9% (Llama3). To beat it, we must match its EM within one point in the Full condition while lowering the generic-hedge rate in the Absent condition.

SBA. SBA is the strongest generator-side method. It reports a 10% absolute F1 gain for LLaMA-3.1 and a 5% accuracy gain for Mistral on CONFACT's GPT-4-identified conflict split (ModC). We use the same background texts under the same context budget.

Hallucination-only DPO in the style of RAG-HAT. This is the strongest tuning competitor. RAG-HAT reports a 26.9% drop in hallucination rate and a 57% win rate on RAGTruth. We train it on identical instances, with the same sample count and DPO hyperparameter sweep, using $\gamma = 0$ and no split between credible and non-credible documents. This baseline doubles as the key ablation. The ablation that drops only the $\mathrm{Dep}$ term (keeping credibility weighting) isolates our main ingredient. We predict that removing it lowers $N_\ell$ and $N_u$ but increases hedging.

Chain-of-Specificity prompting. This is the specificity-maximizing contrast. We expect it to raise the unsupported-claim count $N_u$ in the Absent condition.

No hedging baselines exist yet. We found no reported result on Lacuna for generic-hedge rate under credibility interventions, for any of these methods. Those numbers will be produced here.

Why not just…?

Why not just prompt the model to "be specific, but only use credible sources," on top of SBA? This is the most serious threat, especially for strong instruction-following models. The evidence against it is indirect. CrAM finds that models use credibility scores placed in the prompt poorly without training. The Bertsch line of work finds that citation markers alone mislead models. Neither result tested specific abstention, however. We therefore run this prompt as a first-class baseline at every model size. If it reaches our frontier, the contribution narrows to the diagnostic and the $\mathrm{Dep}$ metric, and we will report it that way.

Why not just combine CrAM with Chain-of-Specificity? One method suppresses bad sources and the other pushes for detail, so the pair seems to cover both failures. Our hypothesis is that the combination is exactly what produces parametric confabulation. Once dubious tokens are down-weighted in the Absent condition, the pressure for specificity has nowhere grounded to go. The combination is included, and its $N_u$ in that condition decides the question.

Why not just rerank sampled outputs by $R$ at inference instead of training? Best-of-$n$ reranking with the same reward is a real competitor. It may capture most of the gain at a cost of $n$ times the inference compute. We report it at $n$ equal to the number of samples used to build the DPO pairs. If reranking matches the tuned model, the lesson is that the reward matters more than the training.

Why not use conformal prediction sets for trustworthiness? At 90% coverage TRAC already needs 20.4 answers on Natural Questions. That is generic output at the level of the set, and it inherits whatever errors are in the corpus. We include TRAC only as a reference on the TriviaQA conditions and report set size alongside our measures.

Proposed Experiments and Decisive Checks

Diagnostic test of the premise. We measure $N_c$, $N_\ell$, $N_u$, hedge rate (the share of responses with $\mathrm{Dep}$ below a threshold set by human judgments), and EM for every baseline in each evidence condition. The premise holds if two things are true: moving from vanilla RAG to CrAM or SBA raises hedge rate in the Absent condition, and Chain-of-Specificity raises $N_u$ there. If the existing interventions already produce specific abstentions, the premise is false, and we will say so.

Validity of the metric. $\mathrm{Dep}$ must separate human-labeled hedges from human-labeled specific abstentions better than two simpler measures: length and lexical diversity. It must also survive the extractive-copy control.

Main method test. The tuned model must move the joint tradeoff curve outward. That means lower $N_\ell + N_u$ than hallucination-only DPO at equal or lower hedge rate, with EM in the Full condition within one point of CrAM. The frontier is traced by sweeping $\gamma$.

Transfer and side effects. We evaluate transfer, without further training, to the Cochrane summarization set from Evaluating large language models on medical evidence summarization. There we score "certainty illusions," the paper's term for hedged findings restated as absolute claims. Finally, we check jailbreak attack success rate on AdvBench to confirm that the tuning does not reproduce the safety loss reported in the alignment trade-off paper.