Quarantined Introspection: Explanation Telemetry as an Internal Backdoor Tripwire in Defense-in-Depth

When a language model carries a backdoor — a hidden trigger, planted in weights, fine-tuning data, or an agent's memory, that switches the model into attacker-controlled behavior — it passes every clean benchmark and only misbehaves when the trigger appears. Two recent findings sit in uncomfortable tension. Forcing a backdoored model to explain itself exposes the infection: backdoored models show contextual blindness, incoherent justifications, and in about 17% of cases literally name their own trigger as the reason for a prediction, yielding 97.5% zero-shot detection with a GPT-4o judge and 98.8% with internal probe classifiers on LLaMA-3-8B and DeepSeek-7B When Backdoors Speak. Yet the same kind of transparency is a documented liability: counterfactual explanations are optimization-equivalent to adversarial examples Exploring Counterfactual Explanations Through the Lens of Adversarial Examples, and explanations that reveal split thresholds let an adversary extract tree models with 100% fidelity at a fraction of the black-box query cost From Counterfactuals to Trees.

This proposal's core move is to stop treating those two facts as a contradiction by separating computation of explanations from disclosure of them. We build a defense-in-depth architecture in which explanation and activation telemetry — contextual-reliance scores, logit-lens emergence patterns, activation-deviation statistics — is computed inside the trust boundary, consumed by defensive layers, and never released to untrusted callers. This quarantined telemetry becomes a cross-layer tripwire against the hardest current threat: backdoors that persist and propagate through multi-step agent workflows, where memory-stage attacks persist into later steps in 77.97% of cases and tool-stage attacks in 60.28%, with attack success above 90% for memory poisoning BackdoorAgent. The scientific question is whether stacking an explanation-integrity layer with orthogonal monitors survives an adaptive attacker who fine-tunes the backdoor to look introspectively clean — the exact attack the authors of When Backdoors Speak flag as their method's undoing and that nobody has yet built.

Core Claim

The claim has three parts. First, explanation-based backdoor detection fails as a standalone diagnostic but succeeds as one layer in a stack, because an attacker who optimizes the backdoor to produce plausible, contextually-grounded justifications must simultaneously keep its activations inside the clean distribution and its workflow footprint inside normal traffic patterns — and these constraints pull on different parameters, so satisfying all of them should cost the backdoor either stealth or efficacy. Second, the transparency-security tradeoff is an information-boundary problem, not a reason to abandon interpretability: telemetry computed and consumed internally leaks nothing, and we can measure the counterfactual cost by comparing model-extraction and evasion success for attackers with and without telemetry access. Third, in agentic pipelines the right place for this tripwire is the agent boundary, where a flagged explanation-integrity violation can quarantine a message before it re-enters another agent's context and halts the cross-stage propagation that makes memory backdoors so lethal BackdoorAgent].

Formally, the adaptive attacker solves the constrained problem of keeping every layer statistic inside its clean quantile:

$$\min_{\theta_{\text{bd}},\,t}\; \mathcal{L}_{\text{trigger}}(\theta_{\text{bd}}, t) \quad\text{s.t.}\quad s_k(x) \le \tau_k \ \text{for all layers } k \text{ and clean inputs } x,$$

where $\theta_{\text{bd}}$ are backdoor parameters, $t$ the trigger, $s_k$ the $k$-th defensive statistic (explanation-coherence score, activation Mahalanobis deviation, or workflow-anomaly score), and $\tau_k$ its clean-distribution threshold. With one layer this is easy; with $K$ orthogonal layers the feasible region shrinks roughly multiplicatively, which is the defense-in-depth orthogonality argument made concrete — the same additivity demonstrated for refusal-loss-landscape detection stacked with prompt defenses in Gradient Cuff.

Research Program

We build three things. The first is the quarantine layer itself: a runtime module that, for each agent message, computes (a) a contextual-reliance score — does the model's justification actually attend to the input context, or only to its own generated tokens, the pathology When Backdoors Speak calls contextual blindness; (b) a logit-emergence-timing statistic from a tuned lens, since backdoored predictions emerge late and abruptly; and (c) a cheap hidden-state Mahalanobis deviation in the spirit of CIFT, which achieves 0.998 AUROC at under 1ms of latency Caught in the Act(ivation). All three are computed on private deliberation tokens and discarded after scoring; nothing crosses the API boundary. When any statistic breaches threshold at an agent boundary, the message is quarantined rather than propagated, targeting the cyclic context re-injection that drives the 77.97% memory-attack persistence BackdoorAgent]. The disclosure boundary follows the design logic of When to Think, When to Speak, which separates private think tokens from public speak commitments — but our gating signal is security telemetry, not semantic entailment.

The second deliverable is the adaptive attacker: backdoors fine-tuned with auxiliary losses that penalize each defensive statistic directly — a differentiable coherence proxy to suppress contextual blindness and tattling (the open attack noted in When Backdoors Speak), plus activation regularization toward the clean hidden-state distribution, mirroring the activation-space evasion threat already flagged for CIFT Caught in the Act(ivation). This is the "explanation-preserving backdoor," the generative-model generalization of the explanation-preserving attacks that motivate the certified attributions of EnsembleSHAP, which achieved 0.892 trigger-word recall on IMDb against 0.491 for standard Shapley — but our setting drops the ensemble assumption and moves to autoregressive agents, which that work explicitly does not cover.

The third deliverable is the leakage measurement: a controlled experiment quantifying what telemetry disclosure costs. Attackers attempt extraction and trigger-inversion with and without access to the quarantined signals, on the logic that faithful explanations act as regional labels over input subspaces Models That Are Interpretable But Not Transparent. This turns the transparency-security tradeoff from a qualitative worry into a measured curve: detection benefit of the telemetry versus extraction benefit to whoever sees it.

Baselines

The do-nothing reference is the undefended backdoored agent, whose behavior is already quantified: on the BackdoorAgent benchmark, memory attacks (PoisonedRAG, AgentPoison) exceed 90% attack success and persist across stages in 77.97% of cases, with tool-stage persistence at 60.28% and planning at 43.58% BackdoorAgent. We reproduce these numbers on our own model stack (Llama-3-8B, matching When Backdoors Speak) so every comparison shares identical triggers, poisoning rates, and compute.

The strongest existing competitor is BEAT, the black-box defense that concatenates a known-harmful probe with the suspect input and measures the collapse of refusal behavior via Earth Mover's Distance, reporting an average AUROC of 99.6% across Llama-3, Mistral-7B, and GPT-3.5-turbo Probe before You Talk. We run BEAT with the same budget of probe queries our stack uses, on the same backdoored models, tuned on the same validation split. BEAT is the bar to clear because it needs no white-box access at all, which is a deployment advantage our stack concedes; beating it means matching or exceeding its single-turn AUROC on non-adaptive backdoors and holding detection above 80% AUROC under the adaptive explanation-preserving backdoors where we expect BEAT's refusal-collapse signal to degrade, plus halting cross-stage propagation in the agent setting where BEAT, being single-turn, offers no mechanism at all.

The strongest transparency-based competitor is the logit-lens probe classifier of When Backdoors Speak, which reaches 98.8% accuracy, run standalone as a single-layer detector on identical data. The decisive ablation is the stack with and without its key ingredient — the explanation-integrity layer — holding the activation monitor and workflow checks fixed. If removing the explanation layer costs little under the adaptive attacker, the proposal's central mechanism is wrong and the activation monitor was doing the work; if removing it collapses adaptive-attack detection, the quarantined explanation signal is earning its latency.

Why not just…?

Why not just use BEAT and skip the white-box machinery entirely? BEAT's signal is the collapse of refusal when a harmful probe is concatenated with a triggered input Probe before You Talk, which presupposes the backdoor's target is a safety-refusal override. A backdoor whose triggered behavior is a benign-looking structured tool call or a subtly wrong plan has no refusal to collapse, and the sample-dependent-target problem is a documented open limitation of backdoor defenses for generative models. This is a genuine threat to the proposal, and Experiment 2 settles it head-on: we run BEAT on both refusal-target and non-refusal-target backdoors, adaptive and not, and report where its 99.6% holds and where it breaks. If BEAT survives everything we throw at it, that is a finding against us and we will say so.

Why not just never generate explanations — no traces, no leak, no problem? Two cited results block this. Reasoning models generate internal traces whether or not anyone sees them, and those traces actively surface dormant harmful logic in roughly 80% of cases where the final output is a polite refusal CoT is Not the Chain of Truth — the signal exists regardless, so refusing to read it wastes it. And the counterfactual-adversarial equivalence result Exploring Counterfactual Explanations Through the Lens of Adversarial Examples shows the danger is disclosure, not computation. The alternative that does attack the computation directly — poisoning reasoning traces to defeat distillation, as DOGe does at a 39% student-accuracy cost on CommonsenseQA DOGe — deliberately corrupts the telemetry we need, so it works against detection rather than for it.

Why not just find the backdoor and cut it out — fine-pruning or surgical head edits? Surgical editing genuinely works when you know where the flaw lives: updating 0.64%–1.95% of attention-head parameters improved HarmBench safety by 55.0 points with negligible capability loss CoT is Not the Chain of Truth, and fine-pruning exploits the fact that backdoor logic hides in dormant subspaces that clean validation never activates Fine-Pruning. But both assume you already suspect a backdoor and can afford retraining, and trigger inversion does not scale to 32,000-token vocabularies. Our runtime tripwire is complementary: detection is what tells you that and where — the layer that fires localizes the anomalous computation — and quarantine buys time until surgery is possible. We treat post-hoc removal as a downstream consumer of our output, not a rival.

Proposed Experiments

Experiment 1 establishes the non-adaptive setting. We implant backdoors across attack families — weight poisoning, in-context demonstrations following Universal Vulnerabilities in Large Language Models, continuous prompts following BadPrompt, and memory poisoning from the BackdoorAgent suite (Agent QA, Agent Code, Agent Web, Agent Drive) — then measure attack success after defense, clean-task accuracy, false positive rate, and a new propagation-halt rate: the fraction of triggered messages stopped at the first agent boundary rather than reaching a second stage. Layer-by-layer attribution of catches, in the manner of the four-tier defense that cut multi-agent injection from 31.2% to 4.2% at ~225ms cost Beyond Single-Model Injection, tells us which layer earns its latency. We report our latency budget the same way, targeting the sub-millisecond regime CIFT demonstrates Caught in the Act(ivation).

Experiment 2 is the decisive one: the adaptive attacker. For each single-layer statistic and for the full stack, we fine-tune backdoors under the constrained objective above and plot the attack-success-versus-clean-accuracy frontier. The prediction, drawn from the orthogonality additivity in Gradient Cuff and the poisoning paradox in BackdoorBench — where more aggressive backdoors become statistically easier to spot — is that each additional orthogonal constraint the attacker must satisfy steepens the frontier: single-layer-adaptive backdoors evade their target layer but get caught by another, while stack-adaptive backdoors pay a measurable clean-accuracy or trigger-reliability price. If backdoors can satisfy all constraints for free, the proposal fails.

Experiment 3 measures the tradeoff directly: attackers with query budgets attempt extraction and trigger inversion with and without telemetry disclosure, using the regional-label logic of Models That Are Interpretable But Not Transparent. The deliverable is the two-sided curve — detection gained by the telemetry versus attack capability gained by seeing it — with quarantine as the point that captures the first benefit at zero disclosure cost. All backdooring and defense code will be released through the tooling conventions of the SCLBD/BackdoorBench repository so results slot into its evaluation format, extended from its 8,000-evaluation vision-scale grid to the agentic regime where, as its own findings show, fixed-target assumptions break down.