Risk-Adaptive Verification Against Semantic Stealth

Safety filters for language models face a scissors problem. On one blade, attackers now wrap harmful instructions in fluent, natural-looking text — semantic camouflage — so that input-level guardrails see benign prose while the model underneath decodes a harmful task; SemanticCamo reports an average attack success rate of 80.35% across six major models this way. On the other blade, the obvious defense — give every request more thinking time — demonstrably fails: Get RICH or Die Scaling shows that uniformly scaling test-time compute buys zero robustness unless the model was already hardened, and Chain of Risk shows that longer reasoning traces actually leak more unsafe content than final answers do. This proposal argues that the way out is to stop spending verification effort uniformly, and to spend it where a model-internal risk signal says it matters.

The core claim: a defense built from programmatically checkable safety rules (the verifiable-reward idea), whose training rollouts and test-time verification calls are allocated by a probe that reads the model's own internal states, is robust to semantic camouflage in a way that neither uniform verifier-based training nor uniform test-time scaling can be — and it does so at a fraction of the verification cost. The reason is architectural. Semantic camouflage works by exploiting the gap between surface text and deep processing; signals read from the model's own hidden states inherit the model's interpretation of the prompt, not the attacker's choice of words. The LLM Already Knows shows those states carry linear, step-zero signals of whether the model will succeed or struggle, and GAVEL shows mid-to-late layer activations catch misdirection attacks where GPT-4's text-level recall collapses to 12%. No one has yet connected these internal risk signals to the allocation of verifiable safety compute.

Core Claim and Distinction from Closest Work

Three prior lines each hold one piece. Beyond Oracle replaces subjective LLM-judge safety evaluation with synthesized Python unit tests and trains with group-based RL against those deterministic verifiers — but it allocates rollouts uniformly and never tests semantic camouflage, only syntactic conflicts. interwhen (interwhen) compiles natural-language rulebooks into runtime process verifiers that fork and roll back a reasoning trace mid-generation — but its monitoring is reactive, invoked the same way for every prompt, and it never studies an attacker who disguises the intermediate state variables it extracts. VIP (Adaptive Rollout Allocation for Online RLVR) proves that a prompt's gradient variance under binary rewards is $p(1-p)$, where $p$ is the prompt's success probability, and allocates training rollouts accordingly — but it works in benign math domains, predicts $p$ from text embeddings, and would mispredict a camouflaged prompt as easy, since the camouflaged text sits inside a benign embedding cluster. This proposal is the intersection none of them occupy: verifier-supervised safety training plus adaptive allocation, where the allocation signal is deliberately chosen to be one that semantic camouflage cannot easily spoof — internal hidden-state risk, not surface text.

Research Program

The system has three components built around an open-weight reasoning model (DeepSeek-R1-Distill-Qwen-7B and Llama-3.1-8B-Instruct, matching the model families used in the closest prior work, since the probes need white-box access — a real deployment limitation we accept and state plainly).

First, a verifier compiler following the recipes of Beyond Oracle and interwhen: safety rulebooks (instruction hierarchy, tool-use restrictions, prohibited-content policies) are translated into executable Python checks over structured state variables extracted from the reasoning trace. Verification is deliberately process-level — checking intermediate steps, not just the terminal answer — because Chain of Risk shows unsafe content concentrates mid-trace, before the sanitized final output.

Second, a risk probe. A linear probe on step-zero hidden states (per The LLM Already Knows) predicts a per-prompt safety-failure probability $\hat{p}_i$, supplemented during generation by the CUSUM entropy-change detector of Alshaalan and Rodrigues, which localizes fluent adversarial content in 79.6% of alarms where windowed perplexity manages only 17–46%. Crucially, the probe is trained adversarially: we fit it on attacks that were explicitly optimized, AutoDAN-style, to evade it, so that evading the probe is as costly as evading the safety policy itself.

Third, the allocator, at both training and inference time. Training rollouts follow the variance logic of VIP, with the allocation

$$G_i \;\propto\; \hat{p}_i(1-\hat{p}_i) \;+\; \lambda\, r_i,$$

where $G_i$ is the rollout group size for prompt $i$, $\hat{p}_i(1-\hat{p}_i)$ is the predicted gradient variance from the binary safety verifier, $r_i$ is the probe-predicted residual risk (uncertainty about the prompt's safety, not just its expected outcome), and $\lambda$ trades off learning signal against coverage of dangerous prompts. The $r_i$ term is the correction VIP lacks: it forces rollouts onto prompts whose text looks easy but whose internal representation reads as anomalous — precisely the camouflaged case. At test time, prompts with low risk get plain single-pass generation with a DRQA-style token cap (DRQA); high-risk prompts get full process verification with rollback, spending the expensive oracle calls only where they pay. This is the generate-rank-verify economics of ADAP — which showed adaptive search is 2.9x to 5.5x cheaper than the best fixed budget on math and coding — redirected to the safety domain, where its monotonicity assumption is exactly what semantic attacks violate and internal signals are meant to restore.

Baselines

The naive reference point is the unmodified reasoning model: on StrongReject, Beyond Oracle reports DeepSeek-R1-Distill-Qwen-7B at 77.4 before any safety training, and this is the number our do-nothing arm should reproduce. The strongest existing competitor is Beyond Oracle itself — verifier-supervised GRPO with uniform rollouts, which raised StrongReject from 77.4 to 82.2 and IHEval safety on the Llama-8B variant from 16.7 to 36.5. It is the strongest because it is the only prior method that both uses deterministic safety verifiers and evaluates instruction-hierarchy robustness on the same benchmarks we target. To make the comparison fair, we run it with the same total rollout budget as our adaptive method, the same verifier suite, the same base models, and the same tuning effort; beating it means achieving equal or better StrongReject/IHEval safety while reducing total verification calls and rollout tokens by at least 30%, and holding up under the camouflaged attack suite where its uniform training may have under-sampled the hard cases. A second baseline is an off-the-shelf guard model, Llama Guard applied to every input, whose known limits are documented — AUPRC 0.635 on ToxicChat against 0.755 for gradient-based detection in GradSafe — and whose per-request cost is the latency burden Alshaalan and Rodrigues quantify as reducible by 17–22% with selective invocation. The key ablation replaces our adaptive allocator with uniform allocation at matched total compute: same trained verifier, same probe, fixed $G$ for every prompt and fixed verification on every request. If the adaptive arm does not beat this ablation on safety-per-token, the allocation idea is dead and we will say so. A second ablation swaps the hidden-state probe for VIP's text-embedding Gaussian process, isolating whether internal signals are what make the signal camouflaged-robust.

Why not just…?

Why not just run a guard model over every input prompt? The literature is unambiguous that this fails on exactly our threat model: SemanticCamo achieves 80.35% average attack success through abstract feature decomposition that defeats the OpenAI Moderation API, and GAVEL documents GPT-4's recall collapsing to 12% under misdirection where activation monitoring holds above 85%. We include Llama Guard as a baseline anyway, and our decisive experiment would show the probe-and-verifier pipeline holding recall where the guard model collapses on the same camouflaged suite.

Why not just filter on perplexity? Perplexity filters are the exact target semantic stealth is built to defeat — the attacks are fluent by construction. On the CPD benchmark, where adversarial and benign prompts are perplexity-matched, windowed perplexity triggers inside the adversarial content only 17–46% of the time versus 79.6% for sequential entropy detection (Alshaalan and Rodrigues). We adopt that stronger entropy signal as a component; the perplexity filter is simply dominated.

Why not just scale test-time compute uniformly — longer CoT or Best-of-N majority voting? This is the most tempting alternative and the evidence against it is threefold: Get RICH or Die Scaling shows uniform compute scaling yields zero or negative robustness gains for models not already adversarially hardened; Chain of Risk shows extended traces are less safe than final outputs, so more thinking amplifies the leak; and AutoDAN-Reasoning shows attackers scale their own compute too — beam search over attack strategies bought a 15.6-point ASR gain on Llama-3.1-70B-Instruct. A static uniform budget loses an arms race. Our experiment directly tests this: a fixed-budget Best-of-N arm at matched total tokens must be evaluated on the same adaptive attack suite.

Why not just fine-tune on demonstrations of refusing camouflaged prompts? Scaling Test-Time Compute Without Verification or RL is Suboptimal proves verifier-free SFT suffers a suboptimality gap that grows with response diversity — the "copycat trap" — while verifier-based methods achieved 8x compute and 6x data efficiency on Hendrycks MATH; and Beyond Oracle shows standard DPO lagging verifier supervision by 8.2 points on task execution because soft preferences cannot enforce non-negotiable rules. Safety refusals are precisely the high-diversity setting where SFT averaging hurts. We include an SFT-on-refusals arm to confirm this empirically.

Why not just run Beyond Oracle with four times the rollouts? This is the most serious threat, and we treat it as such: if uniform 4x rollouts on the same verifier suite matches our adaptive method's safety at the same total compute, the allocation ingredient adds nothing. Our matched-budget comparison against exactly this configuration is the decisive check. We expect it to fail for two documented reasons: easy syntactic conflicts are mastered quickly ($p \to 1$, so extra rollouts give near-zero gradient, per VIP's variance analysis in Adaptive Rollout Allocation) while camouflaged semantic conflicts — where the gradient signal actually lives — get no more sampling than trivia does; and the test-time side, which Beyond Oracle does not address at all, is where uniform verification either wastes calls on benign prompts or, at low budgets, lets camouflaged ones through.

Proposed Experiments and Decisive Checks

The attack suite combines AdvBench (where SemanticCamo reports its 80.35% ASR), HarmBench (where AutoDAN-Reasoning reports 84.5% ASR on Llama-3.1-70B-Instruct with beam width 8 — our attacker arm, so the defense faces a scaled adversary), the perplexity-matched CPD benchmark of Alshaalan and Rodrigues, and the IHEval/StrongReject hierarchy evaluations from Beyond Oracle. Safety metrics are attack success rate under both static and adaptive (probe-aware) attackers, plus over-refusal on XSTest, where GradSafe reports 0.936 AUPRC as the reference for what internal-signal detection can achieve without blanket refusals. Capability retention is measured on MATH-500 and GSM8K, with the standard set by Chain of Risk: 97.7% of original accuracy retained under a safety intervention. Cost is measured in total tokens, verification calls, and wall-clock, benchmarked against the 2.9x–5.5x savings adaptive over fixed budgets reported in ADAP — no reported result found on Lacuna for verification-call counts in the safety setting specifically, so our measurements will be the first.

The decisive checks are three. First, the matched-budget comparison against uniform-rollout Beyond Oracle: equal safety at ≥30% less verification compute, or the idea fails. Second, the signal ablation: the hidden-state probe must out-predict the text-embedding Gaussian process on camouflaged prompts specifically, since that is the claim that internal signals close the camouflage gap. Third, and hardest, the adaptive adversary: an attacker given white-box access who optimizes prompts to suppress both the risk probe and the monitored verifier states — the evasion scenario GAVEL already flags as its own open problem. If that attacker restores, say, a 60% success rate, the defense still fails at the frontier, and the honest contribution becomes a precise characterization of how internal-signal defenses degrade under targeted evasion — a result the field currently lacks any measurement of.