Domain Mismatch Robust Acoustic Scene Classification using Channel Information Conversion

Seongkyu Mun, Suwon Shon

Abstract

In a recent acoustic scene classification (ASC) research field, training and test device channel mismatch have become an issue for the real world implementation. To address the issue, this paper proposes a channel domain conversion using factorized hierarchical variational autoencoder. Proposed method adapts both the source and target domain to a pre-defined specific domain. Unlike the conventional approach, the relationship between the target and source domain and information of each domain are not required in the adaptation process. Based on the experimental results using the IEEE detection and classification of acoustic scenes and event 2018 task 1-B dataset and the baseline system, it is shown that the proposed approach can mitigate the channel mismatching issue of different recording devices.

Introduction

Acoustic Scene Classification (ASC) aims to identify environments such as "airport," "park," or "metro station" from audio recordings. A significant challenge in real-world deployment is device mismatch. Models trained on high-quality recordings (source domain) often fail when deployed on different hardware, such as budget smartphones (target domain), due to variations in microphone frequency responses and characteristics—collectively termed the "channel." Traditional domain adaptation requires extensive data from every potential target device, which is often impractical. This paper introduces a novel method using a Factorized Hierarchical Variational Autoencoder (FHVAE) to disentangle scene-specific sounds from device-specific channel noise, mapping audio to a "universal domain" to neutralize hardware differences.

Methodology: Channel Information Conversion

The proposed approach leverages the Factorized Hierarchical Variational Autoencoder (FHVAE) to achieve disentanglement. The FHVAE separates the input signal into two independent latent variables:

  1. $z_1$ (Segment-level): Captures rapidly changing acoustic events defining the scene.
  2. $z_2$ (Sequence-level): Captures stable characteristics of the recording device and room impulse response.
The FHVAE architecture (left) and the channel conversion concept (right) using mean shifting of $z_2$.

The core innovation is "mean shifting." By taking the $z_2$ component of either source or target audio and shifting it toward a "universal domain" mean—derived from a large, diverse dataset—the audio is effectively "re-recorded" in a standard, device-agnostic format before classification. This unsupervised process does not require explicit device labels.

Experimental Setup

The study utilized the DCASE 2018 task 1-B dataset, designed for testing device mismatch. It includes:

  • Device A: High-quality mobile recording rig (Source domain, ~24 hours).
  • Devices B and C: Consumer smartphones (Target domains, ~2 hours each).

External data from DCASE 2016 and 2017 challenges were used to establish the "universal domain."

Results and Discussion

Latent Space Disentanglement

Visualization via t-SNE confirms the effectiveness of the FHVAE. As shown in Figure 2, the $z_1$ latent variable is largely device-invariant, whereas $z_2$ exhibits strong clustering based on the recording hardware. This validates the model's ability to isolate "what" is being heard from "how" it is being recorded.

t-SNE plots showing that $z_1$ (left) is device-agnostic, while $z_2$ (right) clusters by device.

Classification Performance

The proposed universal domain conversion significantly improved performance on mismatched target devices (B and C), raising average accuracy from a baseline of 45.6% to 53.7%. This method outperformed popular Generative Adversarial Network (GAN) based adaptation, which typically requires more target-domain data. Interestingly, shifting specifically to the target device's mean (when available) further increased accuracy to 54.5%, demonstrating the flexibility of the latent shifting approach.

Conclusion

The FHVAE-based channel conversion effectively mitigates device mismatch in ASC by mapping diverse inputs to a neutral universal domain. While a slight performance decrease was noted on the original source device (from 58.9% to 57.6%), the overall robustness gain for unseen devices is substantial. Future work will explore applying this disentanglement framework to speech enhancement tasks.

Summary updated

Abstract

In a recent acoustic scene classification (ASC) research field, training and test device channel mismatch have become an issue for the real world implementation. To address the issue, this paper proposes a channel domain conversion using factorized hierarchical variational autoencoder. Proposed method adapts both the source and target domain to a pre-defined specific domain. Unlike the conventional approach, the relationship between the target and source domain and information of each domain are not required in the adaptation process. Based on the experimental results using the IEEE detection and classification of acoustic scenes and event 2018 task 1-B dataset and the baseline system, it is shown that the proposed approach can mitigate the channel mismatching issue of different recording devices.
BibTeX
@inproceedings{mun2019domain,
  author = {Seongkyu Mun and Suwon Shon},
  title = {{Domain Mismatch Robust Acoustic Scene Classification using Channel Information Conversion}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=3oFpZ41ufD},
}
Figure