A sound classifier trained on recordings from one microphone often fails on another. The reason is physical: every microphone acts as its own fixed filter, boosting some frequencies and cutting others, and networks learn that filter along with the sound. When unlabeled audio from the new device is available, the natural fix is self-training. The model labels the new recordings itself, keeps the labels it is confident about, and retrains. Under device mismatch, though, those confident labels are often wrong, and wrong in a structured way: the whole new device gets pulled toward whichever class "sounds like" its filter.
We propose to exploit that structure instead of fighting it. We use a prototype classifier, which assigns each recording to the nearest class "prototype" (the average embedding of that class). A microphone change is time-invariant and acts on every sound the same way, so after frequency-wise normalization we hypothesize that most of the remaining device effect is a single offset in embedding space shared by all classes. If so, the target device's prototypes are the source prototypes plus one shared shift and small class-specific corrections. Every unlabeled target clip, whatever its class, then helps estimate the shared shift. Pseudo-labels are assigned only after the shift is removed, using a class-balanced assignment that cannot collapse the device onto one class. The claim is concrete and testable. Factoring the device effect into a shared component plus a class residual makes pseudo-labeling under hardware mismatch more accurate than estimating per-class target prototypes independently, and more accurate than simply re-centering the data. The same factoring should help much less when the shift is not hardware-like, for example geographic shift.
Where this sits
Three bodies of work supply the pieces. Each leaves the same gap.
Frequency-wise normalization is the established way to remove device coloration without any target data. Relaxed Instance Frequency-wise Normalization normalizes each frequency row of a spectrogram over time and blends this with layer normalization. It showed through mutual-information analysis that device information lives mostly in per-frequency statistics. Residual Normalization adds an identity shortcut to the same idea and won DCASE 2021 Task 1A. Learnable filterbanks found that heart-sound features cluster by stethoscope rather than by pathology, and replaced fixed Mel filters with trainable filters. These papers motivate the proposal by showing the shift is largely a frequency-axis filter. They also constrain it: all are pure generalization methods with linear classifier heads, so none uses unlabeled target audio. Dummy Prototypical Networks is the closest audio work combining prototypes with this normalization. It studies few-shot open-set keyword spotting and never updates prototypes from unlabeled data.
Prototype-guided pseudo-labeling supplies the adaptation machinery. Transferrable Prototypical Networks (TPN) pseudo-labels target images above a fixed 0.6 confidence threshold, builds a separate target prototype per class, and pulls source and target prototypes together. ALPHA filters CLIP pseudo-labels by distance to the assigned prototype and to the nearest competitor. PGPL anchors text embeddings to prototypes and lowers thresholds for under-represented classes. Confident Sinkhorn Allocation (CSA) replaces greedy thresholding with optimal transport, an assignment that respects target class proportions and so prevents class starvation.
All four treat the target shift as unstructured. Each class's target prototype must be estimated from that class's own noisy pseudo-labels. This is exactly where rare or acoustically diffuse classes fail.
Audio self-training under shift shows the failure mode. The ICME 2024 semi-supervised acoustic scene challenge baseline uses multi-stage hard pseudo-labeling with a linear head. It reaches 59% macro accuracy on CAS 2023 but only 29% on Public Squares. Its shift is geographic, not hardware.
The proposal differs from TPN-plus-normalization, the obvious combination, in one specific way. It does not estimate the target prototypes as K independent quantities. It estimates one shared device shift from all target clips, plus heavily regularized class residuals.
Core Claim
Let $c_k$ be the source prototype of class $k$ (mean normalized embedding of labeled source clips). For an unlabeled target device $d$, model its prototypes as
$$c_k^{d} = a_d \odot c_k + \delta_d + r_k^{d},$$
where $\delta_d$ is a shared offset vector and $a_d$ is a per-dimension scale (initialized to one). Both are common to all classes of device $d$. The class residual $r_k^{d}$ is penalized toward zero with weight $\beta$, and $\odot$ is elementwise product.
The parameters are fit by alternating two steps. In the assignment step, target clips are soft-assigned to the shifted prototypes by Sinkhorn optimal transport, with an estimated class marginal $\hat\pi$ that is itself re-estimated each round. This keeps a whole device from being dumped onto one class. In the update step, the shift is recovered in closed form from all assigned clips,
$$\delta_d = \frac{1}{N_d}\sum_{i} \Big( z_i - \sum_k q_{ik}\, (a_d \odot c_k + r_k^{d}) \Big),$$
where $z_i$ is the embedding of target clip $i$, $q_{ik}$ its transport assignment to class $k$, and $N_d$ the number of target clips. Residuals are updated only from high-margin clips.
Because $\delta_d$ averages over every clip rather than one class, its estimation error shrinks with $N_d$ rather than with the per-class count. This is the statistical advantage the claim rests on. Two points distinguish it from simply subtracting the target mean. Subtracting $\sum_k q_{ik} c_k$ separates the device shift from differences in class mix. Fitting $a_d$ handles the gain changes that normalization leaves behind. The resulting assignments then serve as pseudo-labels for fine-tuning the encoder with a prototype-distance loss, and the shift is re-estimated after each round.
Research Program
The work has three stages.
The first stage is a diagnostic that decides whether the idea is viable at all. On TAU Urban Acoustic Scenes 2020 Mobile, every device has labels. We train a frequency-normalized encoder on device A and compute true per-class prototypes on each other device. We then decompose the prototype differences $c_k^{d} - c_k$ into the component shared across classes and the class-specific remainder, and report the fraction of variance explained by the shared part. We do this with and without normalization, and at several encoder depths. If, after normalization, the shared part explains little, the factored model has no basis and we stop or reformulate. We expect it to explain most of the remaining shift for the simulated devices S1–S6, which are generated by impulse-response filtering. We expect it to explain less for the real phones B and C, and little for the CAS 2023 geographic shift. That last case serves as a planned negative control.
The second stage builds the training-free version: frozen encoder, alternating shift estimation and transport assignment. We compare it against the simpler corrections described below.
The third stage closes the loop. The encoder is fine-tuned on the transport pseudo-labels, and the shift is re-estimated each round. We track pseudo-label precision per class and per device across rounds, since rising precision is the direct measure of reduced confirmation bias.
Baselines
The naive reference is a source-only model. It uses the ResNorm or RFN backbone trained on labeled source devices, evaluated on the target device with no use of unlabeled audio and either a linear or a nearest-prototype head. On TAU 2020 Mobile, Residual Normalization reports 75.3% accuracy for its compressed 61.0 KB model. RFN reports "over 10%" improvement on unseen devices over standard baselines. We found no per-device accuracy on Lacuna for either method, so we retrain both ourselves under our splits and report our own per-device numbers alongside the published aggregate.
The cheapest adaptation competitors are two forms of re-centering, both run on exactly the same unlabeled target pool. Input-level spectral equalization subtracts the target device's mean log-spectrum and adds the source mean. This is the textbook correction for a linear time-invariant filter. Embedding-level centering subtracts the target embedding mean and adds the source embedding mean. These are the most dangerous baselines, because if the device shift really is a shared offset they may capture most of it.
The strongest existing prototype method is TPN, re-implemented on the same normalized backbone. It keeps its fixed-threshold per-class target prototypes and prototype-alignment losses. Its reported 80.4% on VisDA 2017 is a vision result and does not transfer. The strongest existing linear-head method is the challenge-style multi-stage self-training of the ICME 2024 baseline, run with RFN and with CSA-style Sinkhorn allocation in place of a fixed threshold. On CAS 2023 we compare directly to the reported 59% macro accuracy and 29% Public Squares accuracy of SE-Trans. On heart sounds, the reference is the 80.91% modified accuracy of the Type II learnable-filterbank model on PHSDB-test from Humayun et al.. No reported result was found on Lacuna for the unseen-device HSSDB set, so we establish that number ourselves.
All methods share the backbone, the labeled and unlabeled data, the number of adaptation rounds, and the same hyperparameter search budget. Selection uses a held-out labeled device that is never the evaluated target.
The ablations isolate the key ingredient. We remove the shared offset, leaving per-class residuals only, which is essentially TPN with transport assignment. We remove the class residuals, leaving a shared offset only. We replace transport with greedy nearest-prototype assignment. We remove frequency-wise normalization. Beating the baselines means improved macro accuracy on the held-out target devices, averaged over seeds with confidence intervals. The gain over both re-centering baselines and over TPN must be largest on the hardest simulated devices and on rare-class subsampled pools.
Why not just…?
Why not just use frequency-wise normalization and skip adaptation? RFN and ResNorm strip static per-frequency statistics. They cannot use target audio, and the field's own summary lists truly unseen microphones as still degrading. The diagnostic stage measures how much shift survives normalization. If the residual is negligible, the source-only baseline will match us and we will report that.
Why not just equalize the spectrum or re-center embeddings using the unlabeled target mean? This is the real threat, and the proposal is built to answer it. Both corrections are special cases of our model with no scale, no class residual, and the target class mix assumed equal to the source mix. We test that assumption directly by subsampling target pools to skewed class proportions. Naive centering then absorbs the class-mix difference into the "device shift" and should degrade, while the jointly estimated offset should not. If re-centering matches us on balanced and skewed pools alike, the transport and residual machinery is unnecessary, and we will say so.
Why not just run TPN or ALPHA on a normalized backbone? Both estimate each class's target prototype from that class's own pseudo-labels, so classes with few or noisy target clips get poor prototypes. That is the Public Squares pattern seen in the ICME challenge. The shared-offset-removed ablation is exactly this method, and comparing it against our full model on rare classes settles the question.
Why not just use linear-head self-training with Sinkhorn allocation? CSA already fixes class starvation. What it lacks is a model of the shift, so its confidences remain biased by the device filter. Running it on the same normalized backbone and pools, and comparing pseudo-label precision per round, shows whether modeling the shift adds anything beyond balanced allocation.
Proposed Experiments
The primary benchmark is TAU Urban Acoustic Scenes 2020 Mobile. We use leave-one-device-out targets over B, C, and S1–S6. Device A is always a labeled source, and the target device's training clips serve as the unlabeled pool. Reported quantities are per-device and macro accuracy, pseudo-label precision per round, and the shared-variance fraction from the diagnostic.
The second hardware benchmark is heart sounds: train on PHSDB, adapt on unlabeled HSSDB recordings, and report modified accuracy.
CAS 2023 is the negative control. There, geography rather than hardware drives the shift, so we predict the shared-offset gain over TPN-style per-class prototypes will be small. A large gain there would mean the method's advantage is not specific to hardware.
Three results would decide the proposal. First, the shared component must explain most of the post-normalization device shift on TAU. Second, the full model must beat both re-centering baselines on class-skewed target pools while roughly tying them on balanced pools. Third, it must beat the per-class-prototype ablation most on rare and diffuse classes. If the first fails, the hypothesis is wrong. If the second fails, the cheap correction is enough. Either outcome is worth reporting.
References
- Domain Generalization with Relaxed Instance Frequency-wise Normalization for Multi-device Acoustic Scene Classification
- Domain Generalization on Efficient Acoustic Scene Classification Using Residual Normalization
- Towards Domain Invariant Heart Sound Abnormality Detection Using Learnable Filterbanks
- Dummy Prototypical Networks for Few-Shot Open-Set Keyword Spotting
- Transferrable Prototypical Networks for Unsupervised Domain Adaptation.
- Prototype-Guided Pseudo-Labeling with Neighborhood-Aware Consistency for Unsupervised Adaptation
- Prototype-Guided Pseudo Labeling for Semi-Supervised Text Classification
- Confident Sinkhorn Allocation for Pseudo-Labeling
- Description on IEEE ICME 2024 Grand Challenge: Semi-supervised Acoustic Scene Classification under Domain Shift