Description on IEEE ICME 2024 Grand Challenge: Semi-supervised Acoustic Scene Classification under Domain Shift

Jisheng Bai, Mou Wang, Haohe Liu, Han Yin, Yafei Jia, Siwei Huang, Yutong Du, Dongzhe Zhang, Dongyuan Shi, Woon-Seng Gan, Mark D. Plumbley, Susanto Rahardja, Bin Xiang, Jianfeng ChenarXiv 2024 · DOI · arXiv

Introduction

Acoustic Scene Classification (ASC) is a fundamental task in audio signal processing, aimed at identifying the environment in which a sound was recorded—such as a busy street, a quiet park, or a bustling restaurant. While significant progress has been made in device-invariant ASC, a new frontier has emerged: geographical domain shift. This paper introduces the IEEE ICME 2024 Grand Challenge, which addresses the performance degradation encountered when models trained on data from one region (e.g., Europe) are deployed in another (e.g., China).

The Challenge of Geographical Domain Shift

Geographical domain shift occurs because the "acoustic signature" of a scene is deeply tied to local culture, language, architecture, and daily habits. A model trained exclusively on European acoustic data may achieve high accuracy locally but fail significantly when tested on Chinese recordings. Preliminary findings show that a model trained on European data achieved a mere 14% accuracy in Chinese environments, highlighting a massive "culture gap" in audio data.

Conceptual overview of geographical domain shift between Europe/Korea and China.

Beyond the shift itself, the field faces a "data scarcity" bottleneck. Labeling audio data is labor-intensive and expensive. To combat this, the challenge promotes the use of semi-supervised learning to leverage vast amounts of unlabeled audio data.

Semi-Supervised Methodology

The proposed baseline system utilizes a multi-stage semi-supervised framework designed to adapt a general model to a specific local domain using limited labeled data.

The semi-supervised learning pipeline involving pre-training, fine-tuning, and pseudo-labeling.

The process follows four key steps:

  1. Pre-training: An initial SE-Trans model is trained on the TAU UAS 2020 dataset.
  2. Fine-tuning: The model is refined using a small set of labeled data from the target Chinese cities.
  3. Pseudo-labeling: The model generates "pseudo-labels" for a large pool of unlabeled Chinese audio.
  4. Final Training: The model undergoes a final training phase using both the original labeled data and the high-confidence pseudo-labeled examples to improve robustness.

The architecture, SE-Trans, integrates Squeeze-and-Excitation (SE) blocks to emphasize critical features and a Transformer encoder to model long-term temporal dependencies in the audio signal.

The CAS 2023 Dataset

The challenge is built upon the Chinese Acoustic Scene (CAS) 2023 dataset, which provides a distinct cultural profile compared to previous European-centric datasets.

  • Scale: Over 130 hours of audio across 10 scene categories from 22 Chinese cities.
  • Development Set: Contains 4.8 hours of labeled data and 19.3 hours of unlabeled data from 8 cities.
  • Evaluation Set: Includes 3 hours of data from 12 cities, 5 of which are "unseen" during training to rigorously test domain adaptation.

Results and Scene-Specific Insights

The baseline system achieved a macro-average accuracy of 59% on the evaluation set. However, performance varied significantly across different environments. Scenes with distinct, rhythmic, or consistent sounds, such as Metro (90%) and Bar (87%), were classified with high precision. Conversely, complex outdoor environments like Public Squares (29%) proved difficult, as they often lack a unique acoustic signature and blend into a generic mix of voices and distant traffic.

Conclusion

The ICME 2024 Grand Challenge sets a new benchmark for robust ASC. While the baseline provides a solid foundation, the significant performance gap in "unseen" cities and confusing scenes like public squares remains a hurdle. The research community is invited to develop more sophisticated semi-supervised algorithms and domain adaptation techniques to bridge the geographical divide in acoustic scene understanding.

Summary updated

Abstract

Acoustic scene classification (ASC) is a crucial research problem in computational auditory scene analysis, and it aims to recognize the unique acoustic characteristics of an environment. One of the challenges of the ASC task is the domain shift between training and testing data. Since 2018, ASC challenges have focused on the generalization of ASC models across different recording devices. Although this task, in recent years, has achieved substantial progress in device generalization, the challenge of domain shift between different geographical regions, involving discrepancies such as time, space, culture, and language, remains insufficiently explored at present. In addition, considering the abundance of unlabeled acoustic scene data in the real world, it is important to study the possible ways to utilize these unlabelled data. Therefore, we introduce the task Semi-supervised Acoustic Scene Classification under Domain Shift in the ICME 2024 Grand Challenge. We encourage participants to innovate with semi-supervised learning techniques, aiming to develop more robust ASC models under domain shift.
BibTeX
@misc{bai2024description,
  author = {Jisheng Bai and Mou Wang and Haohe Liu and Han Yin and Yafei Jia and Siwei Huang and Yutong Du and Dongzhe Zhang and Dongyuan Shi and Woon-Seng Gan and Mark D. Plumbley and Susanto Rahardja and Bin Xiang and Jianfeng Chen},
  title = {{Description on IEEE ICME 2024 Grand Challenge: Semi-supervised Acoustic Scene Classification under Domain Shift}},
  year = {2024},
  eprint = {2402.02694},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.02694},
}
Figure