Abstract
Deep neural networks (DNNs) often struggle with domain shift in acoustic tasks, where models trained on one device or environment fail in another. This paper proposes Variational Bayesian Knowledge Transfer (VBKT), a framework that adapts models by focusing on deep latent variables rather than the vast parameter space of the network. By treating these intermediate representations as probability distributions and employing variational inference, VBKT achieves state-of-the-art performance in acoustic scene classification and speech command recognition with minimal adaptation data.
Introduction: The Challenge of Domain Shift
Modern acoustic models are highly sensitive to changes in recording conditions. A model optimized for high-fidelity studio audio often experiences a catastrophic drop in accuracy when deployed on consumer devices or in noisy environments. This "domain shift" is a significant barrier to the real-world deployment of deep learning models.
As shown in the table above, standard architectures like ResNet see their accuracy plummet from ~79% to ~37% when moved from a known source device to unseen target devices. Traditional adaptation methods, such as fine-tuning, often require large amounts of labeled data and risk "catastrophic forgetting." While Bayesian methods offer a robust alternative for uncertainty estimation, applying them to millions of DNN parameters is computationally prohibitive. VBKT addresses this by performing Bayesian adaptation on a low-dimensional "bottleneck": the deep latent variables.
Methodology
The core innovation of VBKT is shifting the focus from model weights to deep latent variables ($Z$), the intermediate hidden embeddings produced by the network.
The VBKT Framework
The framework uses Variational Bayes to approximate the posterior distribution of latent variables. It encodes source-domain knowledge into a prior and combines it with target-domain likelihood from a small amount of new data. Two primary estimation strategies are employed:
- Gaussian Mean-Field Variational Inference (GMFVI): Utilized when parallel data (identical audio recorded by different devices) is available.
- Empirical Bayes (EB): Utilized for non-parallel data, estimating hyperparameters directly from the source data.
To further improve performance, the authors introduced VBKT-rela, which uses a relational distance term (Huber loss) to ensure the internal geometry of the latent space—how different classes relate to one another—is preserved during transfer.
Experimental Evaluation
The researchers evaluated VBKT on two distinct tasks:
- Acoustic Scene Classification (ASC): Using the DCASE 2020 dataset, which includes 10 scenes recorded across 9 different physical and simulated devices.
- Spoken Command Recognition (SCR): Using a 10-class subset of the Google Speech Command Dataset v2, with added background noise at various signal-to-noise ratios (SNRs).
Key Results and Insights
- Superior Accuracy: In the ASC task, the
VBKT-GMF-relavariant boosted accuracy from ~35% back up to ~70%, outperforming 13 other cutting-edge transfer methods, including standard Teacher-Student Learning (TSL). - Noise Robustness: In the SCR task,
VBKT-EB-relaachieved 93.36% average accuracy, significantly higher than the "No-transfer" baseline of 53.91%. - Distributional Benefits: Unlike point-estimation methods, VBKT considers the entire distribution (mean and variance), capturing the inherent uncertainty in noisy acoustic data.
- Structural Preservation: The success of the
relavariants demonstrates that maintaining the relative positioning of classes in the latent space is crucial for preventing model confusion during adaptation.
Conclusion
VBKT provides a data-efficient and mathematically grounded framework for acoustic domain adaptation. By focusing on deep latent variables, it bypasses the curse of dimensionality associated with full-parameter Bayesian adaptation. Future work will explore learning target variance dynamically and extending the framework to more complex tasks like full-scale automated speech recognition (ASR).
Summary updated