Withhold, Don't Block: Value-of-Information Disclosure Policies Against Adaptive Attacks

When a machine learning model is defended against adversaries, its performance on ordinary inputs almost always gets worse. Adversarially trained image classifiers lose clean accuracy; safety-aligned language models refuse harmless questions; prompt-injection defenses delete the very text they were supposed to process. The literature treats this "robustness tax" as a fixed frontier: every gain in security is paid for in benign utility. At the same time, a separate literature on multi-agent interaction has established that information is not monotonically valuable once an adversary is in the loop — agents from RL communicators to LLM simulators learn to deliberately go silent, because what you reveal shapes what your opponent does next. These two literatures have not met. This proposal argues they should: the robustness–accuracy trade-off is, at its core, an information-disclosure problem, and the tools of strategic information withholding — deciding which bits to reveal, to whom, at which moment — offer a way to move off the frontier rather than along it.

The concrete setting is adaptive adversarial interaction with language models. An attacker running a search-based jailbreak or a multi-turn manipulation does not passively receive the model's output; the attacker uses it. Each response the model gives is an observation that narrows the attacker's search. Benign users also use responses, but they need far less information per turn than an attacker does: a benign user asking "how do I kill a Python process?" needs one command; an optimizer probing for a jailbreak needs the model to confirm, at each step, that a candidate string is working. The core claim of this proposal is that a defender can exploit this asymmetry. By capping and shaping the information content of each response — disclosing the bits a benign task requires while withholding or coarsening the bits that would accelerate an attacker's search — the model can degrade attack progress much faster than it degrades benign utility, without touching a single weight and therefore without paying the standard clean-accuracy tax.

Core Claim

Prior work shows both halves of this opportunity but never joins them. The robustness side documents that every weight-level or filter-level defense taxes clean performance: adversarial training on CIFAR-10 with a ResNet18 yields 79.8% clean accuracy under standard training versus 90.6% when the adversary is modeled with a specific incentive structure rather than worst-case loss, per Adversaries With Incentives: A Strategic Alternative to Adversarial Robustness — but that gain is bought in parameter space and still assumes the defender knows the attacker's incentive set. On the fidelity side, Security--Fidelity Tradeoffs: The Hidden Cost of Prompt Injection Defense shows that across 48 model-defense configurations no system is simultaneously secure and faithful: the most trusting configuration (Llama 3.3 70B) reaches 96.5% fidelity but only 47.8% security, while the most secure reach 99.3% security at 71.0–73.9% fidelity. Crucially, that paper treats suppression ("Ignore") as a failure mode of defenses; it never considers that deliberate, selective withholding could itself be the optimal policy. Meanwhile, game theory says it would be: Prices of Anarchy, Information, and Cooperation in Differential Games shows that continuous feedback channels raise equilibrium costs for all players, Zero-Sum Games involving Teams against Teams: Existence of Equilibria, and Comparison and Regularity in Information formalizes that opponents respond to the meta-knowledge that information is available, and Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks shows RL agents learn to gate communication shut under competitive pressure. Most suggestively, Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs demonstrates that models already produce inert fake-compliance outputs that halt gradient-based attackers mid-search — but treats this as a pathology that fools safety judges, never as a designed defense, and never measures its effect on clean task performance.

The proposal's claim: a deliberate, test-time disclosure policy — choosing per response among full answer, coarsened answer, clarifying question, and inert plausible-but-unhelpful compliance — can break the security–fidelity Pareto frontier because it changes the interaction protocol rather than the model, leaving clean accuracy untouched, and because it differentially starves the attacker's search of the feedback it needs. No cited work does this: the strategic-training work of Adversaries With Incentives is static in weights and in continuous vision domains; the SECFID work of Security--Fidelity Tradeoffs is passive and single-shot; the strategic-dishonesty work of Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs is emergent and unmeasured on clean utility; and the strategic-learning theory of Information Discrepancy in Strategic Learning assumes linear rules and non-adversarial agents.

The Mechanism, Formally Lightly

Think of each response as disclosing a set of information elements. For element $i$, let $v_B(i)$ be its marginal value to the benign task and $v_A(i)$ its marginal value to the attacker's search progress. A disclosure policy reveals element $i$ only when its benign value dominates its adversarial leakage under a tunable threshold $\lambda$:

$$\text{disclose } i \iff v_B(i) \,>\, \lambda \cdot v_A(i)$$

The policy family interpolates between full compliance ($\lambda \to 0$) and total silence ($\lambda \to \infty$), with the crucial middle regime where domain facts flow while exploit-confirming specifics do not. The stylized theory: an attacker searching a space of $N$ candidate exploits needs roughly $N \cdot 2^{-B}$ expected queries when each response leaks at most $B$ bits of usable feedback, whereas a benign task is completed once cumulative disclosure reaches the task's intrinsic information requirement $I_{\text{task}}$, which is typically small. When attacker demand vastly exceeds benign demand, a per-response information cap strictly improves the achievable security–utility pair — a discrete cousin of the non-monotonicity of measurement quality in competitive duopoly games established by Informational properties of the Nash solutions of two stochastic nonzero-sum games. The design must respect the meta-knowledge trap identified by Zero-Sum Games involving Teams against Teams: the attacker knows the policy exists, so the policy must be evaluated against attackers who explicitly condition on it, not against naive fixed attacks.

Research Program

Three components. First, formalize the repeated disclosure game: a defender with the disclosure policy above, an attacker who adaptively chooses queries (gradient-based search, tree-of-thought probing, or multi-turn decomposition) while observing only the defender's graded outputs, and a benign user drawn from an ordinary query distribution. Characterize when a separating equilibrium exists — one in which benign users complete tasks at near-full utility while the attacker's expected success within a query budget collapses. Second, build the policy concretely on an open-weight LLM (e.g., a Llama or Qwen model in the 7B–70B range): an estimator of $v_A(i)$ trained from attacker-model feedback (adapting the activation-probe methodology of Strategic Dishonesty, which achieved an F1 of 0.96 detecting deceptive calculations on QwQ-32B), an estimator of $v_B(i)$ from task-completion signals, and a response generator that implements the four disclosure levels — full answer, coarsened answer (correct but stripped of actionable specifics), clarifying question, and inert plausible compliance (the deliberate honeypot mode). Third, evaluate against the benchmarks below, with the decisive test being the shape of the utility–attack curve as $\lambda$ sweeps: the proposal predicts benign utility decays slowly while attack success decays steeply; if the two decay in lockstep, the asymmetry does not exist and the idea fails.

A secondary line applies the same lens to prompt injection, where the SECFID taxonomy of Execute/Process/Ignore gives the natural outcome variables: withholding here means processing the data-bearing payload while withholding the imperative framing an injection needs, rather than suppressing the whole payload. The theory of Sharing Information with Competitors and the empirical silence results of What Game-Theoretic Benchmarks Miss: Strategic Silence in Multi-Agent LLMs — where competitive agents sent 52% fewer messages and executed 61–93% of deception through pure silence — motivate treating silence itself as a first-class action rather than a bug.

Proposed Experiments

The evaluation uses the benchmarks compiled in the file. For injection defense and fidelity: SECFID (1,168 instances across Entity Extraction, Entity Counting, Translation, and Document Editing, plus the agentic extension), reporting the Execute/Process/Ignore breakdown and the security–fidelity pair. For jailbreak robustness under adaptive attackers: HarmBench and AdvBench with GCG and TAP attackers, following the protocol of yiksiu-chan/SpeakEasy, which showed on GPT-4o that multi-turn Speak Easy interaction raises TAP attack success from 0.575 to 0.985 on HarmBench — exactly the adaptive regime where static defenses should be stress-tested. HarmScore, which measures output actionability rather than surface refusal, is the right primary metric because a coarsened or inert answer should score low on actionability without counting as a refusal. For the benign side: XSTest (250 safe prompts) and OR-Bench (OR-Bench-Hard-1K in particular), measuring benign compliance — the point is to show that the disclosure policy's clean-side cost is near zero because weights are unchanged. For clean task capability: standard accuracy benchmarks unchanged by construction, verified by pre/post comparison of the frozen base model. A key additional measurement is attacker query consumption: expected number of queries to first successful exploit as a function of $\lambda$, which directly tests the search-starvation mechanism. Finally, the meta-knowledge evaluation: run attackers told the policy's form (per Zero-Sun Games involving Teams against Teams) and verify the policy degrades gracefully rather than collapsing.

Baselines

The do-nothing reference point is the undefended base model. On SECFID, Security--Fidelity Tradeoffs reports that Llama 3.3 70B achieves 96.5% fidelity but only 47.8% security; this anchors the insecure-but-faithful end. The strongest existing competitor is the three-way Direct Preference Optimization alignment (preference ranking Process > Ignore > Execute) introduced in that same paper, which raised processing on an unseen task from 43.2% to 80.6% while retaining security — the best published point on the security–fidelity frontier. It will be run with the identical base model, identical SECFID splits and agentic extension, and a matched tuning budget (matching the DPO compute to the policy's estimator-training compute), but critically also under the adaptive multi-turn attack protocol, which its original evaluation did not include. The second strong baseline is input paraphrasing, which Baseline Defenses for Adversarial Attacks Against Aligned Language Models reports reduced attack success from 79% to 5% on Vicuna-7B with minimal benign degradation — the standout cheap defense, run here with the same secondary model and latency accounted. The proposal's key ablation is a matched-suppression random policy: a control that withholds or coarsens responses at the same overall rate as the learned policy but with the $v_B/v_A$ computation replaced by random choice. If the learned policy does not beat this control on attack success at equal benign utility, the value-of-information ingredient — not mere withholding — is doing no work, and the proposal's central claim is falsified. Beating the strongest baseline means: on SECFID, a security–fidelity pair strictly dominating the DPO point (e.g., ≥99% security with ≥85% processing); on HarmBench under TAP-plus-SpeakEasy, attack success below roughly 0.3 where undefended GPT-4o sits at 0.985; on XSTest/OR-Bench, benign compliance statistically indistinguishable from the unaligned base model.

Why not just…?

Why not just DPO-align on Process over Ignore over Execute? This is the real threat, since it already moved processing from 43.2% to 80.6% on an unseen task. But it is a static preference learned on a fixed distribution of single-shot injections, and Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions shows that simple multi-turn and multilingual decomposition drives attack success above 0.9 on models that look robust single-shot. The proposal's experiment runs the DPO model under exactly that adaptive protocol; if DPO holds up there, the honest conclusion is that static alignment suffices for injection and the disclosure policy's value must come from the jailbreak axis instead. The experiments are built to settle this rather than argue it.

Why not just paraphrase every input? Paraphrasing is cheap and, per Baseline Defenses, effective against single-shot suffix attacks (79% to 5% on Vicuna-7B). But it targets input fluency, not output feedback, and Speak Easy demonstrates fluent multi-turn attacks that blow past it (TAP plus Speak Easy reaches 0.985 attack success on GPT-4o). It also doubles inference cost and, more fundamentally, cannot help when the harm lies in how much the response reveals rather than what the request contains. The proposal's method operates on the response channel, so the two are complementary rather than competing; we will report the combination.

Why not just use a bigger model or chain-of-thought reasoning? The file gives direct evidence this fails. Security--Fidelity Tradeoffs evaluated 48 configurations across 8B–70B and found scale and chain-of-thought move along the Pareto frontier, not off it — chain-of-thought improves security with neutral-to-negative fidelity. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal finds robustness largely independent of parameter count across 33 targets, and OR-Bench: An Over-Refusal Benchmark for Large Language Models documents a 0.89 Spearman correlation between safety and over-refusal across 32 model families, with Claude-2.1 refusing nearly 100% of hard safe prompts. Capacity is not the bottleneck; the interaction structure is.

Why not just train with modeled attacker incentives, as in strategic training? Adversaries With Incentives already recovers much of the lost clean accuracy (90.6% versus 79.8% on CIFAR-10) by replacing worst-case loss with attacker utility sets — but it assumes the defender knows or can infer the incentive set from attack history, it is formulated for continuous image perturbations, and its defense lives entirely in static weights. The proposal needs no attack history because it learns what to withhold from the attacker's live feedback, needs no perturbation geometry because it works on discrete text interaction, and pays no clean-accuracy tax because it never modifies the weights. For the file's listed attack histories, no reported result found on Lacuna for strategic training in text or multi-turn settings.

Decisive Checks

Three outcomes would decide the proposal. First, the sweep of $\lambda$ must produce divergent decay curves: benign compliance near-flat (XSTest and OR-Bench-Hard-1K rates within a few points of the unaligned base) while attack success and HarmScore fall steeply under GCG, TAP, and Speak Easy protocols. Second, the matched-suppression ablation must show the value-of-information computation beats random withholding at equal rates — otherwise the mechanism reduces to crude suppression, which Security--Fidelity Tradeoffs already shows costs fidelity. Third, the informed-attacker evaluation must show the policy survives an adversary who knows it exists; if conditioning on the policy collapses the defense, the result would instead sharpen the meta-knowledge theory of Zero-Sum Games involving Teams against Teams — either way, the experiment returns knowledge rather than a null.