SubSpectralNet - Using Sub-Spectrogram based Convolutional Neural Networks for Acoustic Scene Classification

Sai Samarth R. Phaye, Emmanouil Benetos, Ye Wang

Abstract

Acoustic Scene Classification (ASC) is one of the core research problems in the field of Computational Sound Scene Analysis. In this work, we present SubSpectralNet, a novel model which captures discriminative features by incorporating frequency band-level differences to model soundscapes. Using mel-spectrograms, we propose the idea of using band-wise crops of the input time-frequency representations and train a convolutional neural network (CNN) on the same. We also propose a modification in the training method for more efficient learning of the CNN models. We first give a motivation for using sub-spectrograms by giving intuitive and statistical analyses and finally we develop a sub-spectrogram based CNN architecture for ASC. The system is evaluated on the public ASC development dataset provided for the "Detection and Classification of Acoustic Scenes and Events" (DCASE) 2018 Challenge. Our best model achieves an improvement of +14% in terms of classification accuracy with respect to the DCASE 2018 baseline system.

Introduction

Acoustic Scene Classification (ASC) involves identifying an environment—such as an airport, a busy street, or a metro station—based on short audio recordings. While humans perform this task effortlessly, it remains a complex challenge for machines due to the overlapping and non-stationary nature of environmental sounds.

Current state-of-the-art methods favor Convolutional Neural Networks (CNNs) for their ability to extract features from time-frequency representations like mel-spectrograms. However, standard CNNs often treat these spectrograms as monolithic images. By using kernels that span large frequency ranges early in the network, they risk aggregating information too broadly, potentially washing out subtle, band-specific acoustic "fingerprints." For example, the low-frequency rumble of a metro station is distinct from the high-frequency clatter of a shopping mall.

Methods: The SubSpectralNet Architecture

SubSpectralNet addresses the limitations of global CNNs by processing frequency bands independently before fusion.

The SubSpectralNet architecture featuring sub-spectrogram slicing and a multi-branch CNN structure.

The architecture consists of four primary stages:

  1. Sub-spectrogram Slicing: The input mel-spectrogram is divided into multiple horizontal "crops" or sub-spectrograms representing specific frequency bands.
  2. Multi-Branch Sub-Classifiers: Each sub-spectrogram is fed into a dedicated CNN branch. These sub-classifiers are optimized to identify the scene based solely on their assigned frequency range.
  3. Global Fusion: Feature vectors from the final layers of all sub-branches are concatenated. This combined representation is then processed by a "Global Classification Sub-network" composed of fully connected layers.
  4. Joint Training Strategy: The model employs a multi-task learning approach. The total loss is a weighted sum of the losses from each individual sub-classifier and the final global classifier. This forces each branch to learn highly discriminative features for its specific band.

Key Insights

Statistical analysis reveals that different acoustic scenes are more easily distinguished in specific frequency bands. The authors found that:

  • Band-Specific Discriminability: Some classes, like "airport," are highly recognizable in lower frequencies, while others exhibit unique patterns in mid or high ranges.
  • Regularization via Sub-classifiers: Forcing sub-branches to produce independent classifications acts as a form of regularization, leading to more robust feature extraction and faster convergence.
  • Kernel Size Sensitivity: Increasing kernel sizes in standard CNNs often leads to a drop in accuracy, confirming that preserving local, band-specific details is crucial for ASC.

Key Results

The model was evaluated on the DCASE 2018 Task 1A development dataset, which includes 10-second stereo recordings across ten scene classes.

Test accuracy over training epochs for the global classifier and individual frequency-band sub-classifiers compared to the baseline.

SubSpectralNet achieved a peak accuracy of 74.08%, a significant 14% improvement over the DCASE 2018 baseline. As shown in the training curves, the global classifier consistently outperforms both the individual sub-classifiers and the baseline, demonstrating the effectiveness of the fusion approach. Interestingly, some sub-classifiers (e.g., the high-frequency branch) occasionally outperform the global model for specific scenes, suggesting that global models can sometimes be "confused" by noise in other bands.

Conclusion

SubSpectralNet demonstrates that frequency-band specialization is a powerful tool for acoustic scene classification. By explicitly modeling the unique signatures within sub-spectrograms, the network achieves superior accuracy and efficiency. Future work will focus on automating the selection of sub-spectrogram parameters and incorporating temporal modeling techniques, such as Transformers, to better capture the evolution of soundscapes over time.

Summary updated

Abstract

Acoustic Scene Classification (ASC) is one of the core research problems in the field of Computational Sound Scene Analysis. In this work, we present SubSpectralNet, a novel model which captures discriminative features by incorporating frequency band-level differences to model soundscapes. Using mel-spectrograms, we propose the idea of using band-wise crops of the input time-frequency representations and train a convolutional neural network (CNN) on the same. We also propose a modification in the training method for more efficient learning of the CNN models. We first give a motivation for using sub-spectrograms by giving intuitive and statistical analyses and finally we develop a sub-spectrogram based CNN architecture for ASC. The system is evaluated on the public ASC development dataset provided for the "Detection and Classification of Acoustic Scenes and Events" (DCASE) 2018 Challenge. Our best model achieves an improvement of +14% in terms of classification accuracy with respect to the DCASE 2018 baseline system. Code and figures are available at https://github.com/ssrp/SubSpectralNet
BibTeX
@inproceedings{phaye2019subspectralnet,
  author = {Sai Samarth R. Phaye and Emmanouil Benetos and Ye Wang},
  title = {{SubSpectralNet - Using Sub-Spectrogram based Convolutional Neural Networks for Acoustic Scene Classification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  doi = {10.1109/icassp.2019.8683288},
}
Figure