Audio classifiers often learn the recording room instead of the sound they are supposed to recognize. In one clinical speech study, spectral features separated typically developing children from children with developmental disorders with 92.7% accuracy. Much of that signal turned out to be a 100 Hz electrical hum and the reverberation of the clinic, which the school recordings did not share (Classifying language-related developmental disorders from speech cues). Meanwhile, the wavelet scattering transform keeps being adopted precisely because it captures fine temporal structure. The scattering transform is a fixed, untrained cascade of wavelet filters, magnitude operations and averaging that produces "how fast does the energy in each frequency band fluctuate" features. It is now used for deepfake detection on web-scraped audio, for noisy speech recognition, and for clinical EEG. Its second-order coefficients carry the gains in these applications, and they are also where a room's reverberation should leave its strongest fingerprint. Reverberation smears energy envelopes in time, and smearing is a change in exactly the modulation content those coefficients measure.
This note proposes a simple physical model of that fingerprint and a correction that follows from it. With a squared-magnitude nonlinearity, diffuse reverberation acts on each band's power envelope as a fixed linear filter. Each second-order coefficient therefore gets multiplied by one room-dependent number per (band, modulation rate) pair. That number can be estimated from the signal's own sound offsets and divided out. The result should be a scattering representation whose second-order part is close to blind to the room, while keeping the speaker-dependent modulation structure that a per-recording mean subtraction would destroy.
Core Claim
Let $e_{\lambda_1}(t) = |x \star \psi_{\lambda_1}|^2(t)$ be the power envelope of the signal $x$ in the acoustic band $\lambda_1$, where $\psi_{\lambda_1}$ is the wavelet for that band. For a diffuse reverberant tail, the expected reverberant envelope is approximately the dry envelope convolved with the band's energy decay curve $r_{\lambda_1}(t)$; this is the classical modulation-transfer view of reverberation. A second wavelet $\psi_{\lambda_2}$ is narrowband around modulation rate $\lambda_2$, so the second-order coefficient factorizes:
$$ S_2^{\text{rev}}(\lambda_1,\lambda_2) \approx |\hat r_{\lambda_1}(\lambda_2)|\, S_2^{\text{dry}}(\lambda_1,\lambda_2), $$
where $\hat r_{\lambda_1}$ is the Fourier transform of the decay curve. In the log domain the room becomes an additive offset, $\log|\hat r_{\lambda_1}(\lambda_2)|$, that depends only on the room and the coefficient index, not on the content.
The claim has three falsifiable parts. First, this offset structure holds empirically: within a room, the reverberant-minus-dry log-coefficient differences are consistent across utterances. It should hold better with the squared modulus of Deep Scattering Power Spectrum Features for Robust Speech Recognition than with the standard modulus, because only power envelopes add incoherently under a diffuse tail. Second, uncorrected second-order coefficients leak more room information than log-mel features do. Third, dividing out a blind estimate of $|\hat r_{\lambda_1}(\lambda_2)|$ removes most of that leakage while preserving task accuracy. If the first part fails, the correction has no basis and the project reduces to a leakage audit. That audit is still worth reporting.
Relation to Prior Work
The scattering papers supply the machinery and also motivate the worry. Multiscale Scattering for Audio Classification established the cascade and showed that second-order coefficients cut GTZAN genre error from 28% (MFCC) to 18%. Joint Time-Frequency Scattering for Audio Classification extended it with wavelets along log-frequency and reported 15.8% TIMIT phone error versus 18.3% for MFCCs. Differentiable Time-Frequency Scattering on GPU made the joint version trainable-pipeline friendly. None of these was evaluated under reverberation; we reuse their implementations.
Joy et al. introduced the squared modulus and showed that adding second-order coefficients gives a 19.2% relative WER reduction on Aurora-4 under clean training. Their motivation was noise-floor suppression. We give the squared modulus a second, physical justification: it is the nonlinearity under which reverberation becomes a separable gain. We then test whether that gain, left uncorrected, is a liability.
WST-X Series found that a small averaging scale and second-order coefficients matter for deepfake detection on Deepfake-Eval-2024. That corpus is drawn from more than 80 web sources, so source channel and label may be correlated. We do not claim their result is confounded. We propose to measure how much source identity their features carry.
The confound literature defines the target. Classifying language-related developmental disorders from speech cues is the concrete case of a classifier learning the room. Exploiting Wavelet Scattering Transform for an Unsupervised Speaker Diarization left distant-microphone AMI untested, and that condition is where our model predicts the largest effect.
Adaptive front ends address noise but not room identity. Interpretable Representation Learning for Speech and Audio Signals Based on Relevance Weighting learns to down-weight corrupted bands. Speaker Verification with Adaptive Spectral Subband Centroids adapts band edges per frame. Both act on additive-noise robustness, and both lose some clean-condition accuracy or need labels. Neither models the convolutive room effect in the modulation domain.
Research Program
The work has three parts.
The first part validates the model. We take clean TIMIT and AMI headset (IHM) speech and convolve it with measured and image-method simulated room impulse responses, so that the exact decay curves $r_{\lambda_1}$ are known. For each room, we regress reverberant on dry log second-order coefficients. We then compare the predicted offset $\log|\hat r_{\lambda_1}(\lambda_2)|$ with the observed one, separately for the squared and the standard modulus. We expect the factorization to break down where the modulation rate approaches the wavelet bandwidth and for early reflections that are coherent rather than diffuse. We will map that region explicitly.
The second part builds the correction. For each recording, we estimate per-band decay curves blindly from energy offsets, meaning the free decays after a sound stops, using standard per-band decay-slope fitting on $e_{\lambda_1}$. We then divide each $S_2(\lambda_1,\lambda_2)$ by the estimated $|\hat r_{\lambda_1}(\lambda_2)|$.
Stationary additive components such as mains hum mostly add a constant to $e_{\lambda_1}$ in a few bands. The zero-mean second wavelet removes that constant, so hum lives mainly in first-order and zeroth-order coefficients. We handle it by subtracting a noise-floor estimate taken from silent segments. We will also measure the residual leakage through hum-speech cross-terms rather than assuming it is zero.
We deliberately avoid per-recording mean subtraction as the main method. Mean subtraction would also remove a speaker's average rhythm profile, which may be the very clinical marker of interest. We keep it only as an ablation.
The third part is the audit. For every representation, we train a probe to predict the recording condition: room, microphone, session, or web source. We train the task classifier separately and report both. A representation counts as better only if it lowers condition leakage at equal or better task performance.
Baselines
The naive reference is log-mel filterbank features with per-utterance mean normalization, the standard front end. It removes short stationary channel gains but has no model of reverberant smearing. On Aurora-4 the only reported number for it in the file is the 10.7% WER Mel-filterbank baseline in the relevance-weighting paper.
The strongest existing robust front end in the file is the relevance-weighted filterbank of Agrawal and Ganapathy, which reports 9.6% WER on Aurora-4. We will rerun it with the same acoustic model, training data and tuning budget as every other front end, since its number comes from a different back end. Beating it means matching its WER on Aurora-4 while showing lower room or microphone leakage, or lower WER on the reverberant and distant-microphone conditions.
The strongest scattering baseline is the first-plus-second-order squared-modulus feature set of Joy et al. Their reported result is a 19.2% relative WER reduction over first-order alone under clean training. The file gives no absolute WER for it, so we recompute it. The comparison isolates our correction: identical coefficients with and without division by the estimated room gain.
We also include joint time-frequency scattering with μ-log compression from Lostanlen et al. No reported result was found on Lacuna for it under noise or reverberation.
Three ablations isolate the key ingredient. The first applies the correction with the standard modulus; the model predicts weaker gains. The second corrects first-order coefficients only, which is close to ordinary channel normalization. The third replaces blind decay estimation with per-recording mean subtraction; this should remove room leakage but also hurt the clinical task.
Why not just…?
Why not just apply cepstral mean normalization or modulation-domain band-pass filtering (RASTA-style) to log-mel features? These remove stationary convolutive channels, and in the log domain any zero-mean temporal filter cancels a constant per-band gain. For short channels they are a real threat, and our first-order-only ablation should perform about the same. They do not model reverberation tails longer than a frame. Such tails reshape the modulation spectrum, attenuating fast modulations more than slow ones, rather than adding a constant. The experiment that settles it is the paired IHM-versus-distant-microphone AMI comparison and the simulated-room TIMIT study. If RASTA-filtered log-mel features reach the same room-probe accuracy and task error there, our correction adds nothing beyond interpretability, and we will report that.
Why not just train with reverberation augmentation? Multi-condition training is the strongest practical competitor for task accuracy. Joy et al. show it narrows gaps on Aurora-4, where second-order coefficients still help by 12.2% relative. Augmentation teaches the classifier to tolerate rooms, but it does not stop the representation from carrying room identity. When room and label coincide, as in the clinical data, the classifier is still rewarded for using it. We will train augmented versions of every baseline and report leakage alongside accuracy. If augmented log-mel features show both low leakage and matching accuracy, the threat is confirmed.
Why not just dereverberate the waveform first? Lobdell et al. found in Human speech perception and feature extraction that spectral subtraction helped only with stationary, matched noise, and that its benefit disappeared at low signal-to-noise ratio and under mismatch. Our correction needs only per-band decay estimates rather than full signal reconstruction. Still, we will include a dereverberation preprocessing baseline on the simulated-room data so the comparison is direct rather than argued.
Why not just use a self-supervised speech model? Xuan et al. report that prompt-tuned XLSR alone reaches minDCF 0.4052 on Deepfake-Eval-2024, against 0.3408 when scattering is added. They also note that such models smooth over fine transients. Whether these models encode the room less is an open empirical question. We will include XLSR features in the leakage audit but not tune them.
Proposed Experiments
The validation experiment uses TIMIT and AMI headset speech with simulated and measured rooms. It succeeds if the room offsets predicted from known decay curves explain most of the variance of the observed log-coefficient shifts with the squared modulus, and clearly less with the standard modulus.
The paired-microphone experiment uses AMI headset and single-distant-microphone recordings of the same utterances. We measure the distance between paired features and the accuracy of a microphone probe. We expect uncorrected second-order coefficients to separate the two microphones more readily than log-mel features do, and the corrected coefficients to separate them least.
The confound experiment uses the Interspeech 2013 Autism Sub-Challenge, where the raw-spectral 92.7% figure and the official 60.1% UAR serve as reference points. We first train a room probe, clinic versus school, on each representation. We then report task UAR on held-out data, restricted where possible to subsets that break the room-label correlation. Success means the room probe drops toward chance for the corrected features while UAR on the deconfounded subsets is at least as good as with log-mel features.
The noise-robustness experiment uses Aurora-4, including the channel-mismatched Test Set D. It checks that the correction does not cost WER relative to Joy et al. or the relevance-weighting baseline. Diarization error on AMI distant microphones and on VoxConverse, against the 23.0% and 12.2% DER of Arya et al., checks transfer to an unsupervised task.
Finally, a web-source leakage audit on Deepfake-Eval-2024 asks how well WST-X-style features predict the source site, before and after correction. We report this as a measurement, not a verdict on that paper.
The project fails cleanly in one of two ways. Either the offset model does not hold, or RASTA-filtered log-mel features match the corrected scattering features on both leakage and accuracy in the reverberant paired setting. Either outcome would be reported as the main result.
References
- Classifying language-related developmental disorders from speech cues: the promise and the potential confounds
- Deep Scattering Power Spectrum Features for Robust Speech Recognition
- Multiscale Scattering for Audio Classification
- Joint Time-Frequency Scattering for Audio Classification
- Differentiable Time-Frequency Scattering on GPU
- WST-X Series: Wavelet Scattering Transform for Interpretable Speech Deepfake Detection
- Exploiting Wavelet Scattering Transform for an Unsupervised Speaker Diarization in Deep Neural Network Framework
- Interpretable Representation Learning for Speech and Audio Signals Based on Relevance Weighting
- Speaker Verification with Adaptive Spectral Subband Centroids
- Human speech perception and feature extraction