Who Gets Clipped: Demographic Miss Rates in Noise-Robust Voice Activity Detection

Before a voice assistant or speech recognizer hears anything, a small model called a voice activity detector (VAD) decides which slices of audio contain speech and which are background noise. Whatever it labels as silence is thrown away. Modern detectors are trained on synthetic mixtures: clean recordings from a speech corpus with recorded noise added on top. The noise side of these recipes has been studied carefully. The speech side has not. It is inherited from whichever platform supplied the clean recordings, and those platforms have their own contributor populations. Volunteer audiobook readers sit behind LibriSpeech, adult read speech behind TIMIT, and self-selected volunteers behind Common Voice. Children, older adults, non-native speakers, and breathy or creaky voices are therefore thin or absent in the training data. Nobody has asked whether noise-robust detectors clip these speakers more often at low signal-to-noise ratio. A clipped word cannot be recovered by any later fairness fix in the recognizer.

This note proposes to measure that effect and then explain it. The core claim is that VAD miss rates under noise are concentrated on speaker groups the source platform under-supplies. The proposal also hypothesizes that this disparity splits into two parts. One part is driven by pitch and vocal-tract size, and cheap synthetic augmentation can close it. The other part is driven by voice quality, articulation, and pause structure, and only real speaker coverage closes it. The work would build a demographically labeled noisy VAD test set, train detectors under controlled speaker-pool recipes at fixed hours, and test which data lever closes which part of the gap.

Why this is open

Noise-robust VAD research has moved from physical cues to deep models. Periodicity features from the autocorrelation function, as in the AZR method, search a fixed 2–20 ms pitch-lag window. Higher-order statistics (Górriz et al.) and subband power distances (Pham et al.) exploit statistical properties of speech that noise lacks. Large-scale deep benchmarks such as LibriVAD train transformer and convolutional-recurrent models on LibriSpeech mixed with WHAM!, DEMAND, and babble noise. LibriVAD also showed that the silence-to-speech ratio in training data matters as much as the noise level. Every one of these papers reports error averaged over speakers.

Two results suggest the average hides structure. The first is that Cabral et al. showed that voice quality significantly changes how well glottal-event detectors work. Glottal events are the moments the vocal folds close, and detection degrades on breathy, lax, and creaky phonation. That study used very few speakers and no added noise. The second is that the demographic speech-bias literature has stopped exactly at the front end. Quantifying Bias in ASR found large recognition gaps for Dutch children (WER 35.3% versus 18.4% for teenagers) and older adults, but treats the recognizer as a single block. The Sonos bias dataset found age, ethnicity, and dialect effects on clean read commands. Its authors name noisy versions as future work. The two studies also disagree on the direction of the gender gap, and nobody knows whether segmentation under different acoustics explains part of that disagreement.

The closest prior pairing of VAD and demographics is a DNN that jointly classifies frames as male, female, or silence. It uses gender as an auxiliary label for efficiency and never asks whether misses fall unevenly. Our question is the reverse of theirs.

Core Claim

We claim that, at a single deployed threshold and low signal-to-noise ratio, current detectors miss a measurably larger fraction of speech frames for children, older adults, and non-modal voices than for the adult read-speech population they were trained on. We also claim the gap is not uniform. The child gap should track fundamental frequency, the rate at which the vocal folds vibrate, and formant scaling, and pitch or vocal-tract-length augmentation of adult speech should largely remove it. The older-adult and non-modal-voice gaps should track breathiness, speaking rate, and pause patterns, and should persist under augmentation. If so, they close only when the training pool spans more real speakers with those properties.

This differs from Some Voices are Too Common, which found that speaker count beats demographic label balancing for recognition WER on Common Voice. Our hypothesis is that for a frame-level speech/non-speech decision, raw speaker count matters less than coverage of acoustic covariates. We further hypothesize that some of that coverage can be synthesized and some cannot.

Research Program

The first component is a disaggregated noisy test set. Clean speech comes from Jasmin-CGN and CGN, which give Dutch children, teenagers, adults, older adults, and non-native speakers. It also comes from the Sonos dataset, which gives English speakers with age, gender, dialect region, and ethnicity labels. These recordings are mixed with QUT-NOISE-TIMIT scene noise (cafe, home, street, car, reverberant) from -10 to +15 dB, following the QUT-NOISE-TIMIT protocol. Utterances are concatenated with silence to reach the realistic silence-to-speech ratio of roughly 34% used in LibriVAD-Concat.

Frame labels come from the clean signal, before noise is added. Forced alignment itself can be worse for children, so labels are taken from the agreement of alignment and a clean-signal energy detector. Disagreements are audited by hand on a stratified sample, and the label error rate is reported per group. All demographic comparisons are made within a corpus, never across Dutch and English, so that language and recording-setup differences are held fixed.

The second component is the explanatory analysis. For every speaker we measure median and range of fundamental frequency, a harmonics-to-noise ratio as a breathiness proxy, speaking rate, and pause-length distribution. Following the Sonos methodology of multivariate regression with interaction terms, we fit a mixed-effects logistic model of frame miss. The predictors are group, signal-to-noise ratio, noise type, and these acoustic covariates, with a per-speaker random effect. The question is how much of each group's excess miss rate the covariates absorb.

The third component is controlled training recipes. All arms use the LibriVAD ViT architecture and identical compute. Each draws a fixed number of training hours from Common Voice, which carries self-reported age, gender, and accent, and mixes it with the same noise recipe. The arms differ only in speaker-pool selection: platform-natural sampling, demographic label balancing, maximum unique speakers, and covariate-coverage selection. The last arm picks speakers to span the fundamental-frequency, breathiness, and speaking-rate ranges observed in the test populations. It is crossed with and without pitch-shift and vocal-tract-length perturbation augmentation.

Baselines

The naive reference point is the LibriVAD-trained ViT run as released, with one global threshold. That threshold is chosen to give a fixed false alarm rate, 5%, on the adult reference group of each corpus, and it is never retuned per group. This is what a practitioner deploys today, and the size of the gap it produces is the paper's first result.

The strongest existing detector is the ViT on MFCC features from LibriVAD. It reports a peak average AUC of 0.9710 on LibriVAD-Concat and 0.9090 on the out-of-distribution VOiCES set after scaling to the Large training split. It is the strongest competitor because it is the largest-data, best-reported deep detector and was explicitly tested for distribution shift. We use its published configuration and LibriVAD-Large training data. No reported result was found on Lacuna for any VAD evaluated separately by age, gender, or voice type, so its per-group numbers are new measurements.

Classical detectors are included because they encode physical assumptions that may fail differently. These are AZR, which reports a 24.8% average relative HTER improvement over LTSD on QUT-NOISE-TIMIT (Ghaemmaghami et al.), and the GMM-MFCC detector, which was best among the original QUT-NOISE-TIMIT baselines. Both are run with their published settings.

For mitigation, two baselines are matched to our recipes on hours, architecture, and hyperparameter search budget, with 20 trials each selected on a demographically mixed validation split. The first is adversarial removal of group information from the encoder. The second is multi-task training with a group-prediction head. They mirror the UAI-AT and UAI-MTL comparison in the speaker-recognition bias study, where adversarial training raised EER from 2.5% to 3.9% while multi-task training kept 2.7%.

The ablation that isolates the key ingredient compares covariate-coverage selection against random selection with the same number of unique speakers and hours. If coverage does not beat matched-count random selection, the coverage hypothesis is wrong and the result reduces to the known speaker-diversity effect.

Beating a baseline means a smaller worst-group-minus-reference miss-rate gap at matched false alarm rate, with a 95% interval from bootstrap over speakers, not frames. It also means overall AUC no more than 0.005 below that baseline.

Why not just…?

Why not just report disaggregated error on existing VAD benchmarks? LibriVAD and QUT-NOISE-TIMIT draw their speech from adult read-speech sources and carry no child or older-adult speakers. Disaggregating them can only compare adult men and women. Building the test set from Jasmin-CGN and Sonos is therefore a necessary part of the proposal, not optional polish.

Why not just pitch-shift and vocal-tract-length-perturb adult training speech? This is the most serious threat, and the design tests it directly. If augmentation alone closes every group gap, that is a useful and cheap result, and we would report it as the main finding. Our prediction is narrower. Augmentation should close the gap that fundamental frequency explains, mainly children. It should not close the gaps explained by breathiness and pause structure, because pitch shifting does not create aspiration noise or change timing. The augmentation-only arm against the coverage arm, split by group and covariate, settles this.

Why not just train on more platform data or balance the labels? Common Voice has its own self-selection, so more of it reproduces its skew. Some Voices are Too Common found label balancing ineffective for recognition: female-only training gave 16.9% WER on women versus 16.3% for the reference model. LibriVAD reports diminishing returns from Medium to Large on seen conditions. Both recipes are trained as arms here rather than dismissed.

Why not just lower the threshold, or set one per group? A detector does not know the speaker's age before it detects speech. In the joint gender-VAD model, group inference depended on frames the detector had already accepted. Lowering a single global threshold buys recall with false alarms for everyone. We therefore report miss-rate gaps across the full threshold sweep, as an area-under-gap curve analogous to the fairness-discrepancy curves of the speaker-recognition bias study. This guards against a gap that vanishes at one operating point and reappears at the deployed one.

Decisive Checks

The first check is whether the gap exists at all. At matched 5% false alarm on the adult reference group, at 0 dB and below, we test whether the LibriVAD ViT's miss rate for Jasmin children or older adults exceeds that of CGN adults by more than the per-group label error rate. If it does not, the premise fails and we report a negative result.

The second check is whether the gap matters downstream. On Jasmin-CGN we run a fixed recognizer twice, once on oracle segmentation and once on VAD segmentation, and compare WER. The growth of the child-versus-teen and elderly-versus-adult WER gaps under VAD segmentation gives the share of the recognition disparity reported in Quantifying Bias in ASR that is caused upstream.

The third check is the decomposition. In the mixed-effects model we compare the group coefficient with and without the acoustic covariates, then ask which training arm shrinks each residual. The hypothesis is confirmed only if two things hold together. Augmentation must remove most of the fundamental-frequency-attributable child gap. The coverage arm must outperform both augmentation and matched-count random selection on the breathiness- and timing-attributable older-adult and non-modal gaps. A supporting probe on the Male VQ voice-quality recordings, mixed with the same noise, checks whether the breathy and creaky misses mirror the glottal-detector failures reported by Cabral et al..