Stopping Demographic Drift Between Retrieval Hops with a Small Query Gatekeeper

When a retrieval-augmented language model answers a question that needs several pieces of evidence, it works in a loop. It writes a step of reasoning, uses that step as a search query, reads what comes back, and repeats. A recent finding from medical question answering shows that a single mention of a patient's ethnicity or gender can pull a dense retriever (a search system that matches queries to documents by embedding similarity) toward documents that match the demographic term but are clinically irrelevant. In those systems, answers flip under demographic changes 6x to 10x more often than when the same language model is used without retrieval. In a multi-hop loop, a skewed first retrieval produces a skewed reasoning step, which becomes a skewed second query. Nobody has measured whether the drift compounds this way, and nobody has built a fix that works at the point where it would propagate.

We propose to measure demographic retrieval drift hop by hop in iterative multi-hop retrieval, and to fix it with a small, separately trained model, under 1B parameters, that sits between a frozen large reasoner and the retriever. Before each query reaches the index, the small model rewrites it. It is trained with reinforcement learning to keep a demographic attribute in the query when that attribute is needed to find the right evidence, and to neutralize it when it is not. Because the small model only touches query text, it works with black-box APIs that expose no logits. The central test is whether it beats the obvious fix, deleting demographic words from queries, on questions where the demographic attribute actually matters.

Core Claim

We make two claims. The first is that demographic retrieval shift is not a fixed per-query cost in multi-hop retrieval: divergence in the retrieved sets grows with hop index because each skewed retrieval conditions the next query. The second is that the right place to intervene is the query handed from the reasoner to the retriever. The intervention there must be relevance-sensitive, and a small model trained on counterfactual demographic pairs can make that judgment cheaply for a frozen large reasoner.

The prior work splits into three roles. Two papers motivate the problem. Retrieval Shift as a Source of Demographic Bias in Medical RAG establishes the single-hop phenomenon: flip rates of 18.87% to 34.92% for ethnicity, against under 3% for a standalone model, with 86–97% of failures caused by retrieving different documents. It also observes that systems which use chain-of-thought to write queries leak bias into reasoning before retrieval. It stays single-hop and proposes no mitigation. Mental Health Equity in LLMs uses multi-hop questions to find "amplification points" where bias magnifies, but treats multi-hop reasoning only as a detector and mitigates with few-shot prompts, with no retrieval in the loop.

One paper constrains the design. Bias Amplification in RAG studies adversarial corpus poisoning in single-turn retrieval and defends with random query-vector noise plus fairness prompts. That defense becomes one of our baselines. Our setting differs because the drift is natural rather than planted.

Several papers supply the method. Directional Stimulus Prompting trains a small T5 policy with reinforcement learning, rewarded by the large model's downstream output. REFINER and CTRL train small critics of intermediate steps for math and code. We reuse this small-model-steers-frozen-model pattern, but point it at the retrieval query rather than the final answer. None of these works examined retrieval or demographic content.

Research Program

The first component is measurement. We run an interleaved retrieval-and-reasoning pipeline in the style of IRCoT over multi-hop question sets. Each question appears in matched variants that differ only in a demographic descriptor (ethnicity or gender). We log the query and retrieved set at every hop. Our main diagnostic is the Jaccard overlap between the retrieved sets of the two variants at hop $k$, together with the answer flip rate. Compounding means overlap falls faster than a per-hop independent model predicts.

To make that test precise, we compare against the overlap obtained by re-running only hop $k$'s retrieval with the unperturbed hop $k-1$ context substituted in. Any extra loss in the full chain is then attributable to inherited skew rather than to the demographic term itself. We run this on MuSiQue, which was built to block single-hop shortcuts and annotates the supporting paragraphs for each hop. We also run it on HotpotQA and on Perturbed MedQA from Kim et al., executed in iterative mode. Following the Weakest Link Law paper, which found that a multi-hop chain fails at its least visible fact, we also record where retrieved gold paragraphs land in the context. This separates drift-induced loss of evidence from positional loss of evidence.

The second component is a benchmark split that makes the problem non-trivial. In the invariant split, the demographic descriptor describes the asker or a patient whose answer cannot depend on it, as in Kim et al.'s construction. In the relevance split, we draw HotpotQA and MuSiQue questions whose bridge hop depends on a person's nationality or gender, for example a question that must first resolve which country a director is from. Here the attribute has to survive into the query. A gatekeeper that simply deletes demographic words should fail this split.

The third component is the gatekeeper. It is a T5-large-scale model, matching the 750M policy in Directional Stimulus Prompting. It takes the original question, the reasoner's current reasoning sentence, and the proposed query, and it emits a rewritten query. The large reasoner is frozen throughout. Training uses group-relative policy optimization, the reward-normalized policy-gradient method used by CTRL. For a question $q$ and its demographic counterfactual $q'$, the per-hop reward is

$$R_k = \tfrac{1}{2}\left[\mathrm{Rec}_k(q) + \mathrm{Rec}_k(q')\right] - \lambda\,\big|\mathrm{Rec}_k(q) - \mathrm{Rec}_k(q')\big|,$$

where $\mathrm{Rec}_k(\cdot)$ is the recall of the annotated gold supporting paragraphs for hop $k$ when the rewritten query is issued, and $\lambda$ weights a penalty on unequal treatment of the pair. In the invariant split the gold paragraphs are the same for both variants, so the reward pushes toward demographic-invariant retrieval. In the relevance split the gold paragraphs differ, so the reward pushes toward keeping the attribute. The gatekeeper never sees which split an example comes from.

Baselines

The naive reference point is plain IRCoT with the reasoner's raw queries. Our anchor numbers come from Kim et al.: single-hop flip rates of 18.87% to 34.92% for ethnicity, and under 3% without retrieval. For IRCoT's own accuracy, the file reports only aggregate gains (retrieval recall up 11–21 points and F1 up to 15 points across datasets). We found no reported result on Lacuna for per-hop demographic flip rate or retrieved-set overlap under any multi-hop pipeline. Our own IRCoT runs therefore establish the reference.

The first baseline is keyword and named-entity redaction of demographic terms from every intermediate query. It is the cheapest serious competitor. On the invariant split it is a real threat, since deleting the term may be all that is needed.

The second baseline is the BRRA dual defense: random perturbation of the query vector plus a fairness instruction in the generation prompt. It is the only published defense against demographic bias in a retrieval loop. We found no reported result on Lacuna for this defense against natural, non-adversarial drift.

The third baseline is a fairness system prompt plus few-shot BBQ examples, the mitigation that gave 66–94% bias reductions in Khan et al.

The fourth baseline is a retriever fine-tuned for demographic invariance, using contrastive pairs from the same counterfactual data. This is the fix Kim et al. point toward.

The fifth baseline is an intra-model steering vector, applied when the reasoner is open-weight. It follows Shifting Perspectives, which reported average improvements of 12.8% on BBQ with a 3.9% drop on MMLU for Mistral-7B. That makes it the strongest existing inference-time debiasing method, but it has never been tested in a retrieval loop.

For fairness, every learned baseline gets the same counterfactual training pairs, the same retriever and corpus, the same hyperparameter search budget, and the same frozen reasoner. We report results at matched numbers of retrieval calls, because the gatekeeper adds one small-model call per hop.

Three ablations isolate the key ingredient. One trains the gatekeeper on recall only, without counterfactual pairs. One applies the gatekeeper only at the first hop. One trains on the invariant split only. Beating the strongest baseline means a lower flip rate and a higher retrieved-set overlap on the invariant split, with no worse F1 than plain IRCoT, and at the same time no loss of gold-paragraph recall on the relevance split relative to unmodified queries. Redaction should not be able to meet the second condition.

Why not just…?

Why not just strip demographic words from the queries? On the invariant split this may work nearly as well as anything, and we expect it to; the experiment will show it honestly. It has two weaknesses. Kim et al. find that bias leaks into the reasoning before the query is written, so the rest of the query may already be skewed. More decisively, redaction destroys the attribute exactly where the relevance split needs it. If redaction matches the gatekeeper on both splits, we conclude that the learned model is unnecessary and report that result.

Why not just tell the large model to be fair, or use a thinking model? Shrawgi et al. show that safety refusals erode as task complexity rises: ChatGPT's refusal rate fell from 32.8% to 29.4% across their complexity levels. Siddique et al. report that steering beats prompting and Self-Debias on bias benchmarks. On the other side, Zhang and Meng found that extended-thinking models such as Qwen3-8B-Think resisted misleading cues. That is a genuine threat, so we include a thinking reasoner with a fairness prompt as an additional condition. If it alone closes the gap in per-hop overlap, the gatekeeper's value shrinks to cost savings.

Why not just make the retriever demographic-invariant? This is the strongest alternative. A retriever trained to be invariant cannot tell when the attribute matters, and it does nothing about skew already written into the reasoning step. The comparison on the relevance split, and the comparison of hop-1-only against all-hop intervention, settle the question.

Why not steer the reasoner's activations or logits directly? That route requires open weights. DExperts needs full-vocabulary logits, which APIs typically withhold, and activation transfer across scales (Oozeer et al.) needs internal access. The gatekeeper works on query text alone. Where weights are available, the steering-vector baseline measures what text-only steering gives up.

Decisive Checks

The first check concerns compounding. Per-hop overlap on 3–4 hop MuSiQue questions must decline beyond the substituted-context control. If it does not, drift is a per-query effect, and single-hop fixes in the style of Kim et al. would suffice.

The second check concerns the gatekeeper. It must beat redaction on relevance-split recall while matching or beating it on invariant-split flip rate.

The third check concerns placement. All-hop gating must outperform hop-1-only gating on long chains, which is what a compounding account predicts.

The fourth check concerns transfer. A gatekeeper trained with one frozen reasoner and the MuSiQue corpus should keep most of its effect when paired with a different reasoner and with iterative Perturbed MedQA. This addresses the out-of-domain question that Directional Stimulus Prompting left open.

Throughout, we track answer F1 and MMLU-style general accuracy to catch any damage to reasoning. We also check whether answer flips lean toward stereotype-consistent options, using BBQ-style labeling from Parrish et al., so that "drift" is tied to societal bias rather than to noise.