Domain Mismatch Robust Acoustic Scene Classification Using Channel Information Conversion

Seongkyu Mun, Suwon Shon

TL;DR

This paper introduces a novel method for Acoustic Scene Classification (ASC) that remains accurate even when the recording device changes. By using a Factorized Hierarchical Variational Autoencoder (FHVAE) to disentangle scene-specific sounds from device-specific "channel" noise, the authors can map any audio to a "universal domain," effectively neutralizing hardware differences without needing prior data from the target device.

What Problem Does This Paper Address?

Acoustic Scene Classification (ASC)—the task of identifying environments like "airport," "park," or "metro station" from audio—faces a major hurdle in real-world deployment: device mismatch.

Most models are trained on high-quality recordings from a specific device (the "source domain"). However, when these models are used on different hardware, like a budget smartphone (the "target domain"), their performance plummets because the frequency response and characteristics of the microphone (the "channel") interfere with the scene's acoustic signature. Traditional solutions require a large amount of data from every potential target device to "adapt" the model, which is often impossible to collect in advance.

Methods

The researchers propose a Channel Information Conversion approach using a Factorized Hierarchical Variational Autoencoder (FHVAE).

The core innovation lies in disentanglement. The FHVAE is designed to separate the input signal into two independent latent variables:

  1. $z_1$ (Segment-level): Captures the rapidly changing acoustic events that define a scene.
  2. $z_2$ (Sequence-level): Captures the stable characteristics of the recording device and the room's impulse response.
The FHVAE architecture (left) and the channel conversion concept (right).

Once these factors are separated, the authors perform "mean shifting." They take the $z_2$ component of the source or target audio and shift it toward a "universal domain" mean—a neutral baseline extracted from a large, diverse dataset. This process effectively "re-records" the audio in a standard, device-agnostic format before it is passed to the classifier.

Datasets

The study primarily uses the DCASE 2018 task 1-B dataset, which is specifically curated to test device mismatch. It includes:

  • Device A: A high-quality mobile recording rig (Source domain, ~24 hours of data).
  • Devices B and C: Consumer smartphones (Target domains, ~2 hours of data each).

To establish the "universal domain," the authors also utilized external acoustic scene data from the DCASE 2016 and 2017 challenges.

Key Insights and Ideas

  • Successful Disentanglement: Visualization using t-SNE (Figure 3) confirms that $z_1$ is largely device-invariant, while $z_2$ clusters strongly by recording hardware. This proves the model can successfully isolate "what" is being heard from "how" it is being recorded.
  • Universal vs. Pairwise Adaptation: Conventional domain adaptation usually tries to map Device B directly to Device A. This paper argues for mapping everything to a third, neutral "universal domain" (Figure 5). This makes the system far more robust to "unseen" devices that weren't part of the training set.
  • Unsupervised Learning: The FHVAE learns to separate these factors without needing explicit labels for which device recorded which clip, making it highly scalable.
t-SNE plots showing that $z_1$ (left) is device-agnostic, while $z_2$ (right) clusters by device.

Key Results

  • Significant Accuracy Boost: On the mismatched target devices (B and C), the proposed universal domain conversion improved average classification accuracy from a baseline of 45.6% to 53.7%.
  • Superiority over GANs: The method outperformed a popular Generative Adversarial Network (GAN) based adaptation approach, which struggled with the limited data available for the target devices in this task.
  • Target-Specific Gains: When the model was shifted specifically to the target device's mean (rather than a universal one), accuracy reached 54.5%, showing the flexibility of the latent shifting approach.
Concept of shifting the mean of the device-related latent variable $z_2$ to convert channels.

Limitations and Future Work

The authors observed a slight "performance tax" on the original source device (Device A); its accuracy dropped slightly from 58.9% to 57.6% after the conversion process. This suggests that the reconstruction might lose some fine-grained details that were helpful when the training and testing devices were identical.

For future work, the researchers plan to apply this disentanglement framework to speech enhancement, aiming to separate clean speech from unknown, noisy environments in a similar device-agnostic fashion.

Summary updated

Abstract

In recent acoustic scene classification (ASC) research field, training and test device channel mismatch have become an issue for the real world implementation. To address the issue, this paper proposes a channel domain conversion using factorized hierarchical variational autoencoder. Proposed method adapts both the source and target domain to a pre-defined specific domain. Unlike the conventional approach, the relationship between the target and source domain and information of each domain are not required in the adaptation process. Based on the experimental results using the IEEE Detection and Classification of Acoustic Scenes and Event 2018 task 1-B dataset and the baseline system, it is shown that the proposed approach can mitigate the channel mismatching issue of different recording devices.
BibTeX
@inproceedings{mun2019domain,
  author = {Seongkyu Mun and Suwon Shon},
  title = {{Domain Mismatch Robust Acoustic Scene Classification Using Channel Information Conversion}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=3oFpZ41ufD},
}

Loading related content...

Figure