Self-Describing Text as an Attack on Long-Document LLM Judges

LLM-as-a-judge means using a strong language model to grade other models' outputs. It is now the default way to score long-form writing: multi-chapter stories, research reports, long arguments. The quality these judges are most often asked to assess is global coherence, meaning whether the document keeps its characters, facts, timeline and argument consistent from beginning to end. To check this, the judge has to read the whole document and trust what the document says about its own structure. Writers routinely tell readers how the pieces connect: "as established in Chapter 1," "this confirms the earlier finding," "returning to her original motive." We propose that this sentence-level self-description is the natural channel for indirect prompt injection against judges. Indirect prompt injection is an attack in which instructions or persuasion are hidden inside the material a model reads, rather than in the request it was given. Fake cross-references, recaps that assert consistency, embedded praise, and claims of authoritative origin can each be spread thinly across a long document. Together they can push a judge toward "coherent" even when the underlying story contradicts itself. Because no single sentence carries the attack, defenses that look for one decisive span have nothing to find.

The project builds this attack family on long documents with known ground truth and measures how badly current judges and defenses fail against it. It then tests a specific remedy. The remedy is to remove all self-descriptive language at once, as a group, and re-judge. A verdict that survives this ablation is not being carried by what the text claims about itself. The opening question is whether that group ablation beats the obvious cheaper defenses: paraphrasing the whole document, chunking it, or simply telling the judge to ignore meta-commentary. If it does not, the proposal changes, as discussed below.

Core Claim

The claim has two parts. First, judges grading long documents for coherence can be steered by coherence signposting: distributed, fluent statements that assert consistency rather than demonstrate it. This steering survives defensive prompting and chain-of-thought justification, and it evades defenses built on single-span attribution. Second, a set-level self-description ablation restores most of the lost reliability at an acceptable false-alarm rate on honest documents. That ablation works in two steps: tag reader-addressed and self-evaluative spans, delete them jointly, re-score, and compare.

This differs from the closest work as follows. JudgeDeceiver optimizes an adversarial token suffix that makes a judge prefer a short answer. It reports over 90% attack success on Mistral-7B and Llama-3-8B judges on MT-Bench and LLMBar, but it never touches long documents or discourse-level attacks. TopicAttack and StepJack show that smooth topic transitions and attacks split into innocent-looking steps defeat injection filters. They target agents carrying out tasks, not graders. The Silent Judge shows that benign metadata cues flip judge verdicts while the judge's written reasoning never mentions the cue. That paper motivates our concern that the judge's chain of thought cannot serve as a detector. It does not study adversarial content woven into the document body. None of these papers asks what happens when the thing being graded is coherence and the attack consists of claims of coherence.

Why this attack surface is plausible

Three findings in the file point the same direction. Local Coherence or Global Validity? reports two things. Reinforcement learning on math makes reasoning traces locally tidy while they stay globally invalid. A GPT-4o step-level judge tagged errors with only 57.8% accuracy against human labels. So judges already struggle to separate looking consistent from being consistent. Beyond a Million Tokens finds that resolving contradictions is the weakest long-context memory skill across all models. A judge that cannot reliably find a contradiction is likely to accept the document's own assertion that none exists. The Silent Judge found a cue acknowledgment rate of exactly 0%. Its creative-writing results on LitBench include a provenance-cue verdict shift of +16% (human versus LLM labels) for GPT-4o. Biasing cues therefore reach narrative judging, and they leave no trace in the justification.

There is also a non-adversarial version of the threat. A generator trained with reinforcement learning against a coherence judge may learn signposting on its own, because it is cheap and rewarded. That would make this a reward-hacking problem as well as a security problem.

Research Program

Documents with ground-truth coherence. We generate long stories and reports using plan-then-write pipelines in the style of Re3 by Kevin Yang et al. and DiscoDVT by Haozhe Ji et al.. Both build an explicit outline before writing. The outline gives us a ledger of entity states, events and their order, which serves as ground truth. Report prompts come from Deep Research Bench, used in DiffResearch. For each document we then introduce controlled, verifiable contradictions: a character's state flips, two events swap order, or a reported finding reverses. This yields matched clean and corrupted versions whose coherence difference is known.

The attack family. We add signposting at increasing density and spread to the corrupted versions. The variants are false cross-references, recaps asserting continuity, in-world praise of the work, provenance claims in the style of the Silent Judge, and, for comparison, one explicit instruction to the grader. Signposting is written by an auxiliary model with a fluency constraint so that perplexity filters cannot catch it. We also include a JudgeDeceiver-style optimized suffix where the judge's internal weights are available, so that the distributed attack can be compared directly with the known one.

The defense. A tagger model marks spans that address the reader, evaluate the text itself, or assert a connection without stating new world-content. The judge scores the full document and the jointly ablated document. A large gap flags the verdict, and the ablated score is reported. Removing the spans as a group, rather than one at a time, is the key ingredient. It targets the gap that CausalArmor itself concedes. CausalArmor detects injections by removing one segment at a time and checking whether the harmful action disappears. Its authors acknowledge it misses attacks whose influence is spread so evenly that no single removal crosses the threshold.

There is a cost. Honest documents also contain recaps. Deleting them may lower a true coherence score, and that cost must be measured rather than assumed away.

Baselines

Undefended judge. The simplest reference is the same judge model with a standard rubric and chain-of-thought scoring on the full document. This gives the raw attack success rate. It also gives the clean accuracy that every defense must preserve. We will run GPT-4o, Gemini-2.5-Flash (the judges from the Silent Judge), and open Llama-3-8B and Mistral-7B judges, all with identical prompts.

Defensive prompting. We add a defensive system prompt plus sandwiching: the rubric is repeated after the document, with an explicit instruction to ignore any claims the text makes about its own quality or consistency. This is the cheapest serious competitor. The bar is that our defense must reduce attack success further at equal or lower false flags.

Leave-one-out attribution. CausalArmor's leave-one-out method, adapted so that the "action" is the judge's verdict rather than a tool call, is the strongest existing attribution defense. Our framing predicts it will fail on distributed signposting. The file reports CausalArmor at 0.0–0.6% attack success on AgentDojo and 3.65% on DoomArena. There is no reported result on Lacuna for CausalArmor or MELON applied to LLM judges. We will run it with the same segmenter, the same judge, and a threshold tuned on the same validation split as ours.

MELON-style re-execution. MELON catches injections by re-running an agent with the user's request masked and checking whether the same tool calls occur. Adapted here, the judge re-scores under a neutral or swapped rubric. The idea is that a verdict that does not change with the rubric is being driven by the document. MELON's authors note it does not cover text-only outputs, which is exactly our setting. Including it tells us whether that limitation can be patched cheaply.

Paraphrase normalization and chunked judging. These are the two simple-transformation baselines. The first paraphrases the whole document before judging. The second judges segments in isolation and aggregates the segment scores.

Random-span ablation. This ablation isolates our key ingredient. It deletes the same number of tokens, chosen at random rather than by the self-description tagger. If random deletion recovers as much as tagged deletion, the gain comes from shortening the document, not from targeting self-description.

What counts as winning. On signposted corrupted documents, our defense must restore the judge's ability to tell the clean version from the corrupted one to within a few points of its unattacked level. It must do so while keeping false flags on honest documents below those of leave-one-out attribution and paraphrasing, with all methods held to the same number of judge calls.

Why not just…?

Why not just tell the judge to ignore self-praise and meta-commentary? Prompt-level defenses have repeatedly failed against fluent attacks. TopicAttack keeps over 90% attack success against sandwich defenses and against defenses trained into the model to follow only the intended instructions (StruQ and SecAlign). StepJack's defensive system prompts did not stop split attacks. Signposting is also harder to refuse than an instruction, because "as established earlier" is ordinary prose and not obviously a command. Still, this is our second baseline. If it closes most of the gap on our documents, we will report that as the main finding and drop the defense.

Why not just paraphrase the whole document first? This is the most serious threat to the proposal. Paraphrasing removes optimized token strings, but a faithful paraphrase should preserve a sentence like "this matches her vow in Chapter 1," because that sentence is content-like. An aggressive paraphrase, on the other hand, may smooth over real contradictions. The experiment settles this directly by comparing three quantities under each method: attack success, clean-versus-corrupted discrimination, and false flags. If paraphrasing matches us on all three, the contribution reduces to the attack and its measurement.

Why not just judge chunks independently? Chunking does block persuasion that spans segments. But global coherence is precisely a property that spans segments. A judge reading one chunk cannot see a contradiction between Chapter 2 and Chapter 9. Chunked judging is therefore expected to lose its ability to tell clean from corrupted documents even without any attack. Measuring that loss is part of the evaluation.

Why not just use a perplexity filter or an injection classifier? JudgeDeceiver reports that perplexity detection missed 70% of its attacks on Llama-3-8B, even though those attacks were optimized token strings. StepJack's single-observation injection detectors caught monolithic injections 56% of the time but missed split ones. Our attacks are fluent prose by construction. We include an off-the-shelf classifier only as a sanity check.

Proposed Experiments and Decisive Checks

Main pairwise study. We pair clean and corrupted versions of each long document in pairwise comparisons, with signposting at three densities and three spreads. We report three numbers per condition. The first is attack success: how often the corrupted, signposted version wins. The second is verdict shift rate, following the Silent Judge's metric. The third is cue acknowledgment rate: how often the judge's reasoning mentions the signposting.

First decisive check: an evenness curve. We hold the attack's total length fixed and vary how evenly it is spread. The prediction is that leave-one-out attribution degrades as spread increases while group ablation stays flat. If both degrade together, the bottleneck argument is wrong.

Second decisive check: the cost on honest documents. On honest documents written by plan-based generators that use recaps legitimately, we measure how much the true coherence score drops after ablation. This is the cost our defense imposes when nothing is wrong.

Short-form comparison. We run on LitBench and the ELI5 setup from the Silent Judge. This shows whether signposting matters already in short creative passages. It also lets us compare against JudgeDeceiver's MT-Bench and LLMBar regime, where the reported reference figure is over 90% attack success.

Exploratory reward-hacking test. We fine-tune a small generator against a coherence judge with reinforcement learning, then measure whether signposting density rises during training. We also measure whether ablation-adjusted scores diverge from raw scores. Either result would show the attack emerging without any adversary.