Clean-Label Audio Backdoors That Survive the Acoustic Channel

A speech command model that is backdoored during training behaves normally on ordinary speech but switches to an attacker-chosen label whenever a secret trigger is present. Two things currently make such attacks either detectable or physically useless in the audio domain. Attacks that keep the training data correctly labeled (clean-label attacks, which avoid the obvious "a 'Stop' clip labeled 'Go'" tell that human auditors and loss-based filters catch immediately) have only been made to work in vision, or in audio with hand-crafted tricks like pitch shifting that their own authors report degrade once the trigger must survive playback through a real loudspeaker, a room, and a real microphone. Attacks that do survive that over-the-air channel, like embedding room echoes as triggers, work only by mislabeling data, so they forfeit the stealth that motivated clean-label design in the first place. Nobody has built a poisoning attack that is simultaneously clean-label and robust to the physical acoustic channel, and nobody has measured what such an attack costs in clean accuracy or how existing defenses fare against it once legitimate room reverberation is present in every input.

The proposal is to make that attack concrete and study its trade-offs. The core idea is to fold the acoustic channel — the combination of loudspeaker distortion, room impulse response, and microphone response that any played audio passes through — directly into the optimization that crafts the poisoned training samples, using the same expectation-over-transformation trick that made test-time adversarial audio robust to reverberation. The practical stake is a speech command or speaker recognition system trained on crowdsourced or web-scraped audio: an attacker who contributes a few correctly-labeled clips can implant a backdoor that fires when someone simply speaks, or plays a sound, anywhere in the room, while every poisoned sample passes label audits and loss-based anomaly screening.

Core Claim

I claim that clean-label backdoor poisoning can be formulated through a differentiable model of the acoustic channel, and that doing so is the missing piece that lets physical audio backdoors combine the two properties that currently exclude each other: label-consistent stealth and over-the-air activation. The mechanism adapts the dataset-distillation view of clean-label attacks — perturb target-class training samples so that the gradients they induce during training match the gradients of trigger-bearing samples, leaving the labels untouched — from the spatial-image domain where it was proven to the time-series acoustic domain where it has never been tried. The one display equation below captures the whole design:

$$ \min_{\delta,\,\tau}\;\; \mathbb{E}_{h \sim \mathcal{H}}\Big[\, D\big(\nabla_\theta L(\theta;\, h \!\ast\! (x_i+\delta),\, y_i),\;\; \nabla_\theta L(\theta;\, h \!\ast\! x_j + \tau,\, y_t)\big) \Big] \;+\; \lambda\, R_{\text{audio}}(\delta, \tau) $$

Here $x_i$ is a training clip already belonging to the target class $y_i = y_t$, $\delta$ is a small perturbation added to it, $\tau$ is the trigger signal, $h$ is a channel impulse response (speaker, room, microphone composed as convolution), sampled from a library $\mathcal{H}$ of room impulse responses and hardware transfer functions so the expectation trains the poisoning to survive the channel rather than any single realization, $\nabla_\theta L$ is the training loss gradient on model parameters $\theta$, $D$ measures gradient disagreement, and $R_{\text{audio}}$ keeps the perturbation and trigger within psychoacoustic masking thresholds and a temporal-smoothness penalty suited to waveforms. The perturbation $\delta$ suppresses the clip's natural features so the optimizer latches onto the trigger as a shortcut — the same shortcut-learning lever used by label-consistent attacks in vision — but here the shortcut is one that remains a shortcut after passing through a room.

Two scientific questions ride on this formulation, and both are open. First, the imperceptibility–robustness tension: surviving reverberation and hardware filtering demands signal energy, while human inaudibility caps it, a trade-off established quantitatively for test-time attacks by Qin et al. — whose non-robust perturbations fell from full digital success to 0% under simulated rooms while expectation-robust ones held above 60% — and by the finding in AdvSV that cheap hardware acts as an accidental low-pass filter that strips delicate high-frequency content. Does a psychoacoustically masked perturbation still carry enough energy to suppress target-class features during training, or does stealth neutralize the mechanism? Second, the clean-accuracy footprint: clean-label attacks in vision reported near-100% attack success with minimal accuracy loss, but every defense-side intervention in speech recognition pays a steep utility tax, so the proposal treats clean accuracy degradation as a first-class measured outcome, both for the attack itself and for defenses attempting to remove it.

Research Program

The program has three parts. The first part builds the attack, which I call channel-consistent clean-label poisoning (CCP). It replaces the visual regularizers of the vision formulation of Dao et al. — whose Upwind Total Variation loss is defined on spatial pixel grids and has no audio analogue — with waveform-appropriate penalties: a psychoacoustic masking constraint in the style of Qin et al., where loud speech components mask quieter nearby frequencies from human hearing, plus a spectral-envelope smoothness term. The gradient-matching, parameter-matching, and feature-matching variants from the vision work will all be ported and compared, since the vision results suggest they trade off differently across architectures. The trigger $\tau$ itself is optimized jointly with the perturbation under the channel expectation, using room impulse response libraries (RWCP, AIR, RVB2014, BUT ReverbDB, GWA — the corpora used to build TrojanRoom's synthetic room generator) as the sampling distribution $\mathcal{H}$, with hardware transfer functions that model both a high-fidelity and a budget playback-recording chain, motivated by AdvSV's hardware quality gap finding.

The second part studies the energy frontier, which is where the proposal's main scientific risk lives. A sweep over the masking-threshold scale factor will trace a curve from fully inaudible-but-brittle to robust-but-audible poisoning, measuring attack success over the air at each point, so the field gets an actual measured answer to whether the imperceptibility–channel-robustness trade-off that governs test-time evasion also governs training-time feature suppression, and where the knee of that curve sits. Human listening studies following the protocol of the imperceptible-adversarial-audio work (which achieved 76% of listeners judging the audio identical to clean) will validate the perceptual side rather than trusting PESQ alone.

The third part measures the defense landscape under conditions no prior defense evaluation has used: over-the-air audio. Existing speech backdoor defenses were all validated against digitally injected triggers, but a deployed system hears room reverberation in every legitimate input, and AdvSV showed that once audio has physically propagated, a one-class detector's error rate at separating benign re-recorded speech from adversarial speech jumps from 6.66% to over 30% — suggesting detection stages that separate "triggered" from "normal" audio may false-alarm on ordinary reverberant speech. I will evaluate gradient-norm fine-tuning, online time-frequency purification, STRIP-style input filtering, and Neural Attention Distillation (which the vision clean-label authors themselves flag as effective against their attack) against CCP in both digital and physically played test settings, tracking clean accuracy throughout. This connects the attack program to the robustness-accuracy direction: the interesting result may be that physical-channel backdoors are not just harder to plant but harder to remove without destroying utility, because the defense can no longer assume clean inputs look clean.

Proposed Experiments

All experiments use Google Speech Commands (the SPC-10 and SPC-30 splits used by the speech backdoor literature), with victim models spanning the architectures the literature has already benchmarked on this dataset: ResNet-34 and ResNet-18, an LSTM, and transformer-style keyword models (KWT and EAT, from the PBSM/VSVC evaluation), so results are directly comparable to reported numbers. A speaker-recognition extension on VoxCeleb with the RawNet3 and ECAPA-TDNN models from AdvSV tests generality. The physical evaluation rig follows AdvSV's design: three loudspeakers and three recording devices of differing quality tiers, at fixed distance and orientation, with every triggered and clean clip played and re-captured, giving a nine-combination hardware matrix. The reported metrics are over-the-air attack success rate (the fraction of physically played triggered clips that receive the attacker's label), clean accuracy on untouched test data, poison rate, perceptual quality (PESQ plus human studies), and detectability (loss-based anomaly screening and human audit). The decisive checks are: first, that the channel-optimized clean-label attack achieves high over-the-air attack success (target: within a few points of its own digital success rate, where its channel-free ablation collapses, mirroring the 0%-versus-above-60% pattern Qin et al. observed for test-time perturbations); second, that poisoned samples are not separable from clean ones by loss statistics or casual human audit; third, that clean accuracy drops by no more than about one point; and fourth, the energy-frontier curve showing exactly how much perceptibility must be paid for physical activation.

Baselines

The do-nothing reference point is the unpoisoned model trained on the same data splits with the same architecture and compute budget: on the SCD-10 split with ResNet, the gradient-norm fine-tuning paper reports a clean accuracy of 94.54% before any attack, and our attack is only meaningful if the poisoned model stays within roughly a point of that. The strongest existing competitor is TrojanRoom, which poisons speech command models by generating synthetic room impulse responses and injecting them during data augmentation, and which reports over 92% attack success for speech command recognition in physical environments with less than a 3% drop in benign accuracy; it is the only prior audio backdoor demonstrated to activate over the air at distance, so beating it on combined grounds — matching its over-the-air success while surviving label audit and loss-based filtering, which its mislabeled augmentation-time injections do not — is the bar. It will be re-implemented and run on our exact splits, poison budgets, and physical rig so the comparison shares data, compute, and tuning effort. The strongest clean-label competitor is PBSM/VSVC, which modifies pitch, masking, and speaker timbre and reports over 90% attack success on SPC-10 and SPC-30 with under a 1% benign accuracy drop, but whose authors explicitly note performance degrades in the clean-label plus physical-playback regime; it will be run as the representative heuristic clean-label attack under identical conditions, and beating it means closing that acknowledged clean-label-over-the-air gap at a comparable poison rate. The dirty-label control is CBA, which fuses triggers into audio compression features and reports 99.73% attack success on ResNet-34 at a 5% poisoning rate with 96.69% clean accuracy; running its trigger through our physical rig gives the honest ceiling of what abandoning stealth buys. The key ablation removes the channel expectation from the optimization, training the identical clean-label objective on digital audio only, and predicts a collapse under physical playback; that collapse, if it appears, is the isolated evidence that channel-in-the-loop optimization is the ingredient doing the work rather than the clean-label formulation alone.

Why not just…?

Why not just play a static digital trigger through a loudspeaker? The evidence against this is about as direct as it gets. In vision, shifting a static trigger by 2 to 3 pixels drops attack success from 100% to below 50%, because the network memorizes exact stimulus intensities; in audio, non-robust perturbations that succeed fully in digital playback fall to 0% success under simulated room reverberation, and AdvSV documents that consumer hardware strips the high-frequency components such triggers rely on. Still, this objection deserves an experiment rather than a citation, so the channel-free ablation is precisely this baseline run honestly: the same clean-label objective, digital-only trigger, played through the same rig. If it survives physical playback anywhere near as well as the channel-optimized version, the channel model is not earning its complexity and I would report that.

Why not just use dirty-label poisoning, since it already exceeds 99% success? In a setting with no data auditing, dirty-label is indeed sufficient and simpler, and I do not claim otherwise; CBA's 99.73% success at 5% poisoning shows as much. But the threat model that makes clean-label worth its bi-level cost is exactly the crowdsourced-curated data setting where mislabeled samples are flagged by human audit or appear as loss outliers during early training, which is established both by the general clean-label literature and by the observation that Dao et al.'s clean-label attacks outperformed dirty-label baselines in variable physical conditions, plausibly because forcing shortcut learning produces more generalizable trigger features than memorizing a mislabeled artifact. The honest residual risk is that a dirty-label version of our own channel-robust trigger matches it in over-the-air success; the dirty-label control in the experiments settles this by measuring both stealth (audit and loss-based detection rates) and physical success for the same trigger under both labeling regimes, so the contribution claim can be narrowed if stealth turns out to be the only difference.

Why not just defend afterward with pruning or input filtering? Because the measured utility taxes in speech are severe, and physical deployment makes them worse. Baseline fine-pruning, which prunes presumably dormant neurons, crashes speech command clean accuracy to as low as 21% on this exact benchmark, since audio representations entangle backdoor and clean features densely; the gradient-norm fine-tuning method that fixes this still needs around 10% clean data and degrades below that. Online purification fares little better, dropping accuracy on falsely accused clean inputs to around 60% in the worst conditions, and its detector faces a new problem over the air: AdvSV's one-class detector distinguished re-recorded audio at a 6.66% equal error rate but above 30% once asked to separate benign re-recorded speech from adversarial re-recorded speech, indicating that room acoustics themselves will generate false alarms in trigger-detection stages. I should be candid that Neural Attention Distillation purged the vision version of this attack family, so it is a genuine threat; the defense experiments include it directly, and if attention distillation removes our audio backdoor without a clean-accuracy tax under over-the-air conditions, that will be reported as a defense win rather than argued around.

Decisive Checks

The proposal succeeds if the following four results hold on the shared benchmark: channel-consistent clean-label poisoning achieves over-the-air attack success within a few points of its digital success across the hardware matrix while its channel-free ablation collapses under physical playback; poisoned samples are statistically indistinguishable from clean ones under loss-based screening and human audit; clean accuracy degrades by no more than about one point relative to the 94.54% unpoisoned reference; and the energy-frontier sweep produces a reproducible curve quantifying how much perceptibility physical activation costs, with the psychoacoustic constraint left as the operative stealth mechanism rather than an assumed one. The strongest single falsifier is the ablation: if digital-only optimization of the same objective already survives the room, the differentiable channel model is decoration, and the proposal's claim reduces to a domain port of existing vision work.