Question-Swapped Prior Subtraction for LLM Judges

When a language model grades another model's answer, the grader is supposed to ask one thing: does this answer serve this question well? Injection attacks skip that question entirely. A null model (Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates) returns the same fixed adversarial string for every prompt and still reaches an 86.5% length-controlled win rate on AlpacaEval 2.0. That string was optimized on one dataset and transfers to unseen questions, which tells us something useful. Whatever pushes the judge's verdict in these attacks barely depends on the question being asked.

We propose to turn that fact into a defense. For each item, we run the judge a second time on the same candidate responses but paired with other real questions drawn from the benchmark, and we subtract the verdict the judge still gives. A preference that survives when the question no longer matches cannot reflect how well the answer fits the question. It reflects an injected command, a stylistic taste, or a default verdict. Speech recognition has used this move for years under the name internal language model subtraction. There, the model's built-in text prior, estimated by removing the acoustic input, is divided out so that the evidence dominates the decision.

Core Claim

Define the question-blind prior of a judge as its verdict distribution when the true question $q$ is replaced by mismatched benchmark questions $q'$ while both candidates stay fixed. Our claim is that subtracting this prior removes most of the effect of transferable prompt injection while keeping most of the judge's agreement with humans on clean pairs. The subtraction also gives a contamination-style audit statistic, the share of a model's leaderboard win rate that survives question removal.

For a pairwise judge comparing responses $a$ and $b$, let $\ell(q) = \log P(a \succ b \mid q,a,b) - \log P(b \succ a \mid q,a,b)$ be the judge's verdict log-odds read from its decision-token probabilities. The calibrated score is

$$\tilde{\ell} = \ell(q) - \beta \, \frac{1}{K}\sum_{k=1}^{K} \ell(q'_k),$$

where $q'_1,\dots,q'_K$ are $K$ questions sampled from the same benchmark pool and $\beta \in [0,1]$ is the subtraction weight. For absolute 1–10 grading, as in MT-Bench, $\ell$ is replaced by the expected score under the judge's distribution over score tokens.

The distinctive design choice is to use real, in-distribution mismatched questions rather than an empty prompt. In speech recognition, zeroing the encoder output pushes the decoder into states it never saw in training, which is a known weakness of the estimation method in Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition. Fucci et al. avoid this by substituting an average encoder representation. Swapped questions are our analogue: the judge still sees a well-formed evaluation prompt.

Where This Sits

Three lines of work motivate the proposal.

  • Attacks. Null models (Cheating Automatic LLM Benchmarks) and JudgeDeceiver (Optimization-based Prompt Injection Attack to LLM-as-a-Judge) build injections that work across questions. JudgeDeceiver does this by optimizing against simulated "shadow" competitor responses rather than the real test items. Neither paper asks whether this question-independence can be exploited for defense.
  • Absolute scoring. Is LLM-as-a-Judge Robust? shows that universal adjective strings raise absolute scores from 2.3 to 4.8.
  • Passive bias. Preference Leakage shows judges favor outputs stylistically related to themselves, with a Preference Leakage Score up to 37.1% for Qwen-2.5-14B on Arena-Hard. Its fix, contextual calibration, needs human-labeled pairs and estimates one global correction. Our prior is estimated per item and needs no labels.

The method comes from two places. The density-ratio view of speech decoding (Riley and Rybach; Meng et al.) supplies the subtraction itself. Respond in my Language shows that logit-level prior correction transfers to decoder-only LLMs, raising Llama2-Chat language consistency from 18.1% to 72.6%. That paper uses a separate base model as the prior. We instead derive the prior from the same judge under a counterfactual input, so no second model is needed.

Two results constrain the design.

  • Weight sensitivity. Speech recognition shows that setting $\beta$ too high produces degenerate decisions, so $\beta$ must be tuned on clean data and reported as a full curve.
  • Reasoning traces. LLMBar finds that requiring chain-of-thought can make judges rationalize biased choices. We therefore read the verdict from decision-token probabilities and do not use generated reasoning.

The contamination literature supplies the auditing lens. Estimating Contamination via Perplexity detects memorization by comparing a model's perplexity against counterfactual baselines. Benchmark scores that collapse on fresh items (DyVal) expose shortcut success. We apply the same logic to verdicts. A judge's preference should not be predictable without the question, just as a test answer should not be predictable without the test item.

Research Program

We first build the estimator for open-weight judges where decision-token log-probabilities are available: Llama-3-8B, Llama-3-70B and Mistral-7B, the models attacked in the cited work. We sweep $K \in \{1,2,4,8\}$ and $\beta$ from 0 to 1. We compare three sources of mismatched questions:

  • uniformly random questions from the benchmark;
  • questions from the same category;
  • a null "no question provided" prompt, which plays the role of the zero-vector analogue.

The same-category variant is the hardest test: it keeps topic constant and changes only the specific task.

Second, we test against attacks of increasing strength:

  • the published null-model templates;
  • random-search injections optimized on UltraFeedback, following the null-model paper;
  • JudgeDeceiver;
  • the adjective attacks on SummEval and TopicalChat. For summarization, the "question" is the source document, so we swap documents.

Third, and most important, we run an adaptive attacker. We add a term to the JudgeDeceiver objective that also pushes the verdict down under swapped questions. The attacker must then craft an injection whose effect is specific to the question, which the transfer-based attacks in the literature cannot do cheaply. We measure how much the attack's success and its optimization cost change. If adaptive attacks recover full success with little extra effort, the defense fails, and we will report that result.

Fourth, we use the question-blind win rate as an audit statistic on AlpacaEval 2.0 and Arena-Hard-Auto. We also test whether subtraction lowers the Preference Leakage Score. Kinship bias works through style, which the Preference Leakage paper shows is detectable without the question. We therefore predict that subtraction will reduce it.

Baselines

Unmodified judge (the naive reference point). This is the same judge, prompt and position-swap protocol without subtraction. It sets the attack ceiling:

  • null models reach 86.5% length-controlled win rate on AlpacaEval 2.0, 83.0 on Arena-Hard-Auto and 9.55/10 on MT-Bench (Cheating Automatic LLM Benchmarks);
  • JudgeDeceiver exceeds 90% attack success against Mistral-7B and Llama-3-8B (JudgeDeceiver).

It also sets the clean-accuracy reference on LLMBar.

Strongest clean-accuracy method: Rules + Metrics + Reference prompting. This structured prompt from LLMBar raised GPT-4 accuracy on LLMBar Adversarial from about 73% to 82.8%. It is the strongest published prompt-level way to make judges focus on instruction-following rather than surface appeal. We run it with the same open judges and the same prompt budget, both alone and combined with subtraction.

Beating it means two things together. On clean LLMBar Adversarial, accuracy must match or exceed it. Under the null-model and JudgeDeceiver attacks, success must fall below 20%, where structured prompting alone is expected to remain vulnerable. Those attacks hijack exactly the instruction-following ability that structured prompting relies on, and the null-model paper reports that larger judges are more vulnerable.

Strongest calibration method: contextual calibration. This is the label-based correction from Preference Leakage, which reduced its Error Bias metric from 17.8 to 7.3. We give it the same held-out labeled pairs that we use to tune $\beta$. This holds labeled data constant, so any gain from our method comes from per-item estimation rather than extra supervision.

Detection baselines. We include a perplexity filter, which flags adversarial strings with F1 of 0.7–0.8 in Is LLM-as-a-Judge Robust?. We also include the detection defenses that missed 70% of successful injections in JudgeDeceiver. Both are run on identical attack sets.

Ablations. Two ablations isolate the key ingredient:

  • Constant prior. Swapped questions are replaced by the null-prompt prior, which corresponds to zero-vector estimation in speech recognition.
  • Hard flag. Subtraction is replaced by a rule that flags an item whenever the question-blind verdict is confident. This separates the value of soft correction from the value of simple detection.

We have no reported result on Lacuna for question-swapped or rubric-ablated judge calibration, so no prior number exists for our method itself.

Why not just…?

Why not just filter injected text with a perplexity detector? Null-model templates are fluent natural language, and JudgeDeceiver explicitly optimizes for low perplexity. The file reports that defenses miss 70% of JudgeDeceiver's successful injections on Llama-3-8B, and that perplexity filters reach only 0.7–0.8 F1 even against ungrammatical adjective strings. Our evaluation runs the filter on the same attacks. We expect it to fail on fluent attacks, where subtraction should not depend on how the injection reads.

Why not just harden the prompt with delimiters, sandwich instructions, or structured rubrics? Syntactic guardrails are circumvented by adaptive optimization, and the null-model work shows the attack imitates the judge's own template structure. This baseline is still a real competitor on clean data, so we run it both alone and stacked with subtraction. If structured prompting alone drives attack success near zero, our contribution shrinks to the audit statistic, and we will say so.

Why not just apply a single global calibration, as in contextual calibration? A global correction can remove a constant lean toward "Output (a)". It cannot remove a preference that appears only when a particular candidate carries an injection, because that shift is specific to the item. The constant-prior ablation tests this directly. If it performs as well as per-item subtraction, the per-item claim is false.

Why not just ask the judge to detect manipulation, or swap response positions? Swapping fixes order bias; GPT-4 agrees with itself across swaps nearly 97% of the time on LLMBar. It does nothing against content that wins in either position, and JudgeDeceiver keeps high positional consistency. Asking the judge to self-detect relies on the very instruction-following channel the attack hijacks.

Is subtraction just throwing away real quality signal? This is the most serious threat. Some genuine quality, such as fluency and coherence, survives question removal. The LLMBar Natural split and the human-agreement measurements are designed to expose any such loss. We will report the full $\beta$ curve so readers can see the trade-off between robustness and accuracy rather than a single tuned point.

Decisive Experiments

We use the LLMBar data and code (princeton-nlp/LLMBar), AlpacaEval 2.0, Arena-Hard-Auto, MT-Bench, and SummEval/TopicalChat. Metrics are attack success rate, length-controlled win rate of attacked submissions, LLMBar Natural and Adversarial accuracy, and Preference Leakage Score.

The core claim fails if either of two things happens. One is that, at the $\beta$ that brings null-model success below 20%, LLMBar Natural accuracy drops more than three points below the unmodified judge. The other is that the adaptive JudgeDeceiver variant recovers above 80% success at less than twice the original optimization cost.

The claim is supported if three things hold together:

  • subtraction removes most attack gain across all three judge sizes;
  • it reverses the finding that larger judges are more vulnerable;
  • the question-blind win rate separates null-model submissions from genuine leaderboard entries with near-perfect accuracy.

A final check concerns closed judges, where we only have sampled verdicts. We estimate both verdict terms from ten samples per condition on GPT-4-class judges and test whether the sign of the correction stays stable.