Aligning Only the Tail: Domain Adaptation That Keeps a Detector's False-Alarm Budget

A detector that listens continuously, such as a wake-word spotter, a mosquito monitor, or a voice activity detector, is not judged by accuracy. It is judged at one operating point: how many real events it misses while producing no more than an agreed number of false alarms, for example one false trigger per hour. That budget is set on development data by choosing a score threshold. When the device moves to a new room, microphone, or country, the non-target sounds change, the threshold no longer yields the agreed false-alarm rate, and the budget is broken in a way nobody can see without labels. Adversarial domain adaptation is the standard tool for moving a model to a new environment without labels. It trains the feature extractor so that a second network, the domain discriminator, cannot tell source features from target features. It aligns the whole feature distribution, though, and in a continuous audio stream that distribution is almost entirely easy background. The false-alarm rate at a strict budget depends only on the thin upper tail of the background score distribution, the few non-target sounds that look most like targets.

We propose tail-restricted adversarial alignment. The domain discriminator only sees features of audio that the current detector scores near or above its operating threshold, in both domains. The goal is that a threshold calibrated on labeled source data keeps its false-alarm rate on unlabeled target audio, and that recall at a matched false-alarm rate is higher than with global alignment. Put in two sentences: the false-alarm budget lives in the high-score tail, so we spend the adversarial alignment there and not on the easy background. We then measure whether the source-set threshold still holds in the new domain and how many misses remain at that budget.

Core Claim

Let $s(x)$ be the detector's score for an audio window $x$. Let $\tau_\alpha$ be the threshold chosen on labeled source data so that the source false-alarm rate is $P_S(s > \tau_\alpha \mid \text{non-target}) = \alpha$. The quantity a deployed system silently depends on is the target false-alarm drift

$$ \Delta_\alpha = P_T(s > \tau_\alpha \mid \text{non-target}) - \alpha, $$

where $P_T$ denotes the unlabeled target environment. $\Delta_\alpha$ is determined only by how source and target non-target features compare in the region where $s$ is close to $\tau_\alpha$. Global alignment divides its effort across all features in proportion to their frequency, and background far below threshold dominates that frequency. We claim that restricting the discriminator to a score band around and above the threshold produces a smaller $|\Delta_\alpha|$ than global gradient-reversal alignment, moment matching, or no adaptation. We also claim it produces a lower miss rate at an oracle-matched target false-alarm rate. The second claim matters because a method could reduce drift simply by pushing all scores down.

This claim is meant to be falsifiable. If global alignment already removes drift at strict budgets, or if drift can be fixed equally well by recalibrating the threshold on unlabeled target audio, the tail restriction has no purpose. The experiments are designed to show whether either holds.

Relation to Prior Work

The closest work is End-to-End Domain-Adversarial Voice Activity Detection. It attaches a gradient reversal layer to a SincNet and bidirectional LSTM voice detector and makes features indistinguishable across 11 DIHARD recording domains. A gradient reversal layer passes features forward unchanged and flips the sign of the discriminator's gradient on the way back. On an unseen domain, this reduced detection error rate from 13.4% to 11.8%. That paper motivates ours because adversarial alignment clearly helps across acoustic domains. It also leaves our question open. Detection error rate adds misses and false alarms symmetrically at one fixed threshold, and the paper never asks whether a given false-alarm budget survives the shift.

Domain Aware Training for Far-Field Small-Footprint Keyword Spotting does report at a fixed budget of 1 FA/h. On the DMASH distance corpus it compares pooled training, Deep CORAL (matching feature covariances between close-talk and far-field audio), and distance embeddings. This paper supplies our main benchmark protocol and our strongest non-adversarial baselines. However, its false-alarm rate is fixed with labels on the test condition, so it measures recall at an oracle operating point rather than whether the operating point transfers.

Transfer Neyman-Pearson Algorithm for Outlier Detection is the only work in the file that transfers under an explicit false-alarm constraint. It minimizes misses subject to Type-I error at most $\alpha$ by grid-searching classifiers and filtering on labeled target outliers. It shows that pooled or source-only models can suffer severe negative transfer: Type-II error above 0.60 on NASA POWER, compared with 0.37 for their method. That result constrains our design. Indiscriminate transfer can badly hurt rare-event detection, which is part of why we restrict alignment rather than widen it. Their method also needs labeled target positives, works on tabular data, and learns no representation. Ours needs no target labels and adapts the features.

Two works give reasons to distrust the non-adversarial routes. The successive-refinement keyword spotter of To Wake-up or Not to Wake-up shows that false alarms come from a specific confusable region, speech that resembles the keyword, rather than from background as a whole. It reduced out-of-domain false alarms about 7× by modeling that region explicitly. This supports our focus on the tail. HumBug Zooniverse shows that reweighting the rare mosquito class 10× raises detection but grows the false-positive rate from 2% to about 15%. The budget is fragile and cannot be steered by loss weights alone. Finally, test-time adaptation in AdaKWS updates batch-normalization parameters by selectively minimizing prediction entropy, and raises babble-noise accuracy from 26.44% to 49.56%. It is evaluated on pre-segmented clips by accuracy, so its effect on false alarms in an open-microphone stream is unknown.

Research Program

The detector is a standard small keyword or event network, a BC-ResNet or DS-CNN, producing frame or window scores. During training, source windows carry labels and target windows do not. Each step computes the current source threshold $\tau_\alpha$ from a running quantile of source non-target scores. It then defines a band starting at a lower edge $\tau_\alpha - \delta$. The margin $\delta$ is set so that the band holds a multiple $k\alpha$ of source non-target mass, with $k$ around 5 to 20 and swept as a hyperparameter. A discriminator $D$ receives backbone features $f(x)$ only for windows whose score falls in the band. The feature extractor is trained through gradient reversal:

$$ \min_{f,h}\; \mathcal{L}_{\text{det}} \;-\; \lambda\, \mathbb{E}_{x \in B_S \cup B_T}\big[\ell_D(D(f(x)), d(x))\big], $$

Here $h$ is the detection head, $\mathcal{L}_{\text{det}}$ is the source detection loss, $B_S$ and $B_T$ are the source and target windows inside the band, $d(x)$ is the domain label, $\ell_D$ is the discriminator's cross-entropy, and $\lambda$ is the reversal strength. The discriminator minimizes $\ell_D$. The band is recomputed as $s$ changes, so the alignment follows the moving decision boundary.

The main technical risk is that genuine target events also land in the band. If the source side contains mostly non-targets and the target side contains targets, the discriminator can separate domains by class, and alignment will pull target events toward source background. This is the same negative-transfer mechanism as in the Neyman-Pearson transfer paper. We handle it by resampling source band windows so that their target fraction matches an estimate of the target band's event fraction. The estimate comes from the source-calibrated detector's firing rate on target audio, corrected by its source confusion rates. We test whether this correction is necessary with a dedicated ablation. We also study how the event rate of the target stream affects the method, since band contamination grows with event frequency.

The discriminator exists only during training, so the deployed detector's size and inference cost are unchanged. This matters for the small-footprint devices studied in Towards On-device Domain Adaptation for Noise-Robust Keyword Spotting. We do not claim on-device training.

Baselines

The naive reference is the source-trained detector with a noise-augmented, pooled training set and the source threshold applied unchanged in the target domain. On DMASH at 1 FA/h, the pooled baseline's reported false-reject rate is 1.64% at 1 m and 6.33% at 3 m (Domain Aware Training). For HumBug, the reported unweighted CNN achieves a 39% detection rate at a 2% false-positive rate (HumBug Zooniverse). We will reproduce both before comparing anything.

The strongest existing adaptation methods are global gradient-reversal alignment, reimplemented from End-to-End Domain-Adversarial Voice Activity Detection, and Deep CORAL from Domain Aware Training. The latter reports 4.69% false rejects at 3 m and 1.05% at 1 m at 1 FA/h. The distance-embedding variant EMB2 reports 4.11% and 1.02%, but it runs an extra network at inference, so we treat it as a strong reference with a different cost. We report no DMASH number for global gradient reversal because the file contains none. Global gradient reversal is the most direct competitor: it differs from ours only in which windows reach the discriminator.

We also include threshold recalibration on unlabeled target audio: the threshold is set at the $(1-\alpha)$ score quantile of the target stream, treating all target audio as background. Finally, we include AdaKWS-style selective entropy test-time adaptation, followed by the same recalibration.

For fairness, all methods share the backbone, source data, augmentation, and unlabeled target audio. Each gets the same number of hyperparameter trials, 30, chosen on a held-out labeled validation domain distinct from the test domains. Each is run with five seeds.

Two ablations isolate the key ingredient. The first uses a random-subset discriminator, which sees the same number of windows as the band but drawn uniformly from all audio. This separates "aligning the tail" from "aligning less data". The second removes the event-fraction correction. We count our method as beating the competitors if it reduces absolute false-alarm drift at the source threshold by at least half relative to global gradient reversal. It must also either match or improve the false-reject rate at an oracle-matched target false-alarm rate. On DMASH 3 m, that means below the 4.69% CORAL figure at 1 FA/h. All results must be consistent across seeds.

Why not just…?

Why not just recalibrate the threshold on unlabeled target audio? This is the most serious threat. For very rare events it may work: if targets are far rarer than $\alpha$, the target score quantile is essentially the background quantile. It fails in a known direction as events become common relative to the budget. On HumBug, roughly 10% of clips are positive against a 2% false-positive budget, so the top-quantile threshold is set mostly by real mosquitoes and lands far too high, silently inflating misses. Recalibration also cannot improve recall at a matched budget, because it only moves the threshold. We will settle the question by sweeping the target event rate on synthetic continuous streams and reporting where recalibration breaks. If recalibration matches our drift across the realistic range and our recall gain at the matched budget is negligible, we will report that the tail restriction is not worth its complexity.

Why not just use global adversarial alignment, which already helps? The DIHARD result shows global alignment improves aggregate error. However, aggregate error is dominated by the bulk of the score distribution, and nothing in that paper measures behaviour at strict budgets. Global alignment also mixes classes when event rates differ across domains, which is the negative-transfer pattern documented in Transfer Neyman-Pearson Algorithm for Outlier Detection. The random-subset ablation and a direct DET-curve comparison at 0.1 and 1 FA/h will show whether global alignment leaves drift in the tail.

Why not just train with a false-alarm-constrained or cost-sensitive loss on the source? Such a loss shapes the source operating point, but cannot know where the target background tail sits. HumBug's 2%-to-15% jump under reweighting shows how sensitive the budget is to the loss. We include a source-only constrained baseline, trained with a hinge penalty on source false alarms above $\alpha$, to measure how much drift remains when no target audio is used.

Why not just use test-time entropy adaptation? AdaKWS shows large accuracy gains, but entropy minimization on an open stream that is mostly background can make the model confidently fire on background or confidently suppress everything. That effect is invisible in clip accuracy. We run it under identical stream conditions and report its drift and misses directly.

Proposed Experiments

The primary benchmark is DMASH. Close-talk audio (0.25 m) is the labeled source and 1 m and 3 m are unlabeled targets, evaluated as DET curves of false-reject rate against false alarms per hour. We report drift at the source-set 1 FA/h threshold and false rejects at oracle-matched 1 FA/h and 0.1 FA/h. The second benchmark is HumBug Zooniverse, with the recording sources (UK, Kenya, USA, Thailand) as domains in leave-one-out order. It tests the high-event-rate regime at a 2% false-positive budget. The third benchmark is Google Speech Commands built into continuous streams. The keyword occurrence rate is controlled, and background is MS-SNSD babble, ESC-50 events, and DEMAND noises that were held out from augmentation. This setup allows the event-rate sweep that decides the recalibration question. DIHARD voice activity detection, where speech is common rather than rare, serves as a boundary case. There we expect the tail restriction to matter less, and the results will show whether that holds.

The decisive check is a single figure. For each method, it plots target false-alarm drift at the source threshold against the target event rate, next to false rejects at the matched budget. The proposal succeeds if tail alignment has the lowest drift across event rates and also the best or tied recall. It fails informatively if unlabeled recalibration matches it at realistic event rates, or if the random-subset ablation performs as well. The second outcome would mean the benefit comes from gentler alignment rather than from targeting the tail.