A Hybrid Approach to Acoustic Scene Classification Based on Universal Acoustic Models

Xue Bai, Jun Du, Zi-Rui Wang, Chin-Hui Lee

Introduction

Acoustic Scene Classification (ASC) aims to identify the environment in which an audio clip was recorded, such as a "bus," "park," or "library." This task is challenging because different scenes often share similar acoustic events—for instance, engine hums are common to both buses and trams. Standard deep learning approaches, like Convolutional Neural Networks (CNNs), typically treat audio as a holistic spectrogram image, often failing to capture the underlying "syntax" or sequential correlation of shared segments.

This paper introduces a hybrid framework that treats environmental sounds as a language. By decomposing audio into "Universal Acoustic Units" (analogous to phonemes in speech) and applying text-mining techniques like Latent Semantic Analysis (LSA), the proposed method achieves significant performance gains over standard baselines.

Proposed Methodology

The authors propose a three-stage hybrid approach to bridge the gap between holistic audio processing and sequential event modeling.

The proposed hybrid framework for acoustic scene classification, showing the training and testing stages across acoustic modeling, semantic analysis, and classification.

1. Acoustic Segment Modeling (ASM)

Instead of relying on pre-defined labels, the system automatically discovers "universal acoustic units." Using a Gaussian Mixture Model - Hidden Markov Model (GMM-HMM) framework, audio is segmented into variable-length pieces. Through iterative training, a "dictionary" of these units is created, allowing any audio clip to be transcribed into a sequence of symbols (e.g., Unit M1 followed by Unit M10).

2. Latent Semantic Analysis (LSA)

Borrowing from Natural Language Processing (NLP), these sequences are treated like text documents. Term Frequency-Inverse Document Frequency (TF-IDF) scores are calculated for unigrams and bigrams of the acoustic units. To manage high dimensionality and data sparsity, Singular Value Decomposition (SVD) compresses the data into a lower-dimensional "concept space."

3. Classification

The compressed vectors are fed into a back-end classifier. The study evaluates both Support Vector Machines (SVM) and Deep Neural Networks (DNN) for this final stage.

Experimental Setup

The approach was validated using the DCASE 2018 Task 1A dataset, which contains recordings from 10 different acoustic scenes. The development set includes 6,122 training segments and 2,518 testing segments. Audio was processed into 60-dimensional Mel-frequency cepstral coefficients (MFCC) features.

Key Insights and Results

The core hypothesis is that acoustic scenes possess an internal "syntax" where fundamental events combine in specific patterns. By creating a universal set of units, the model can identify shared components (like engine noise) across different scenes, allowing the classifier to focus on the unique units that differentiate them.

Accuracy results for different numbers of ASM units, showing that 20 units provide the optimal balance between variety and complexity.

Impact of Resolution

The researchers found that the number of ASM units is a critical hyperparameter. As shown in the results, 20 units provided the best accuracy (66.1%). Using too few units (10) failed to capture sufficient variety, while too many (40) led to overfitting and increased computational complexity.

Performance Comparison

The Hybrid-DNN approach achieved an accuracy of 66.1%, significantly outperforming the DCASE 2018 baseline CNN score of 59.7%. Notably, the DNN vastly outperformed the SVM (53.3%) on these LSA-transformed features, suggesting that deep architectures are better suited for finding patterns in the compressed concept space.

SVD as a Noise Filter

Dimensionality reduction via SVD proved beneficial. Reducing the data to 80% of the singular values yielded better accuracy than using 100%, indicating that SVD effectively removes redundant noise from the acoustic sequences.

Conclusion

The hybrid framework demonstrates that treating acoustic scenes as a sequence of discrete units can overcome the limitations of holistic spectrogram analysis. Future work may involve exploring more advanced "text-based" classifiers, such as Transformers, and developing formal "scene language" rules to better define the grammar of environmental sounds.

Summary updated

Abstract

For the acoustic scenes classification, the main challenge is distinguishing similar acoustic segments between different scenes. To solve this problem, many deep learning based approaches have been proposed without considering the relevance of different acoustic scenes. In this paper, we propose a novel acoustic segment model (ASM) for acoustic scene classification. ASM aims at giving finer segmentation and covering all acoustic scenes through searching for the underlying phoneme like acoustic units. Furthermore, acoustic segments are modeled by Hidden Markov Models (HMMs) and each audio is decoded into ASM sequences without prior linguistic knowledge. Similar to the term vector of a text document, these ASM sequences are converted into co-occurrence statistics feature vectors and SVM/DNN is used as classifier back-end. Validated on the DCASE 2018 task, the proposed approach can achieve a competitive performance with single model and no data augment. By using visualization analysis, we excavate the potential similar units hidden in auditory sense.
BibTeX
@inproceedings{bai2019hybrid,
  author = {Xue Bai and Jun Du and Zi-Rui Wang and Chin-Hui Lee},
  title = {{A Hybrid Approach to Acoustic Scene Classification Based on Universal Acoustic Models}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-2171},
}
Figure