SubSpectralNet - Using Sub-spectrogram Based Convolutional Neural Networks for Acoustic Scene Classification

Sai Samarth R. Phaye, Emmanouil Benetos, Ye Wang

TL;DR

SubSpectralNet is a novel CNN architecture that improves Acoustic Scene Classification (ASC) by breaking audio spectrograms into frequency-band "slices" called sub-spectrograms. By training specialized sub-classifiers on these bands and a global network to fuse their insights, the model achieves a 14% accuracy boost over the DCASE 2018 baseline.

What Problem Does This Paper Address?

Acoustic Scene Classification (ASC) involves identifying an environment (like an airport, a busy street, or a metro station) based on a short audio recording. While humans do this instinctively, it is a complex task for machines because soundscapes are a messy mix of overlapping events.

Current trends favor Convolutional Neural Networks (CNNs) because they excel at finding patterns in time-frequency representations (spectrograms). However, standard CNNs often treat spectrograms like standard images, using kernels that aggregate information across the entire frequency range too early. The authors argue that this "global" approach misses the fact that different acoustic scenes have unique "fingerprints" hidden in specific frequency bands—for example, the low-frequency rumble of a metro station versus the high-frequency clatter of a shopping mall.

Methods

The authors propose SubSpectralNet, which moves away from processing the whole spectrogram at once.

  1. Sub-spectrogram Slicing: The input mel-spectrogram is divided into multiple horizontal "crops" or frequency bands.
  2. Multi-Branch Architecture: Each frequency band is fed into its own dedicated CNN branch (a "sub-classifier"). These branches are trained to identify the scene based only on their specific frequency range.
  3. Global Fusion: The features from the final layers of all these sub-branches are concatenated and passed into a "Global Classification Sub-network" (a deep neural network).
  4. Joint Training: Crucially, the model is trained using a combined loss function. It back-propagates errors from both the individual sub-classifiers and the final global classifier simultaneously. This forces the sub-branches to learn highly discriminative features for their specific bands rather than just acting as generic feature extractors.

Datasets

The model was evaluated using the DCASE 2018 ASC development dataset (Task 1A).

  • Content: 6,122 two-channel (stereo), 10-second audio samples for training and 2,518 for testing.
  • Classes: Ten distinct acoustic scenes, including airport, bus, metro, park, and street traffic.

Key Insights and Ideas

  • Frequency Matters: Statistical analysis performed by the authors showed that some classes are much easier to distinguish in specific bands. For instance, the "airport" class is highly recognizable in lower frequencies, while "bus" sounds have a distinct lack of activity in the mid-range.
  • Smaller is Better (Sometimes): The authors found that increasing the kernel size in standard CNNs actually decreased accuracy. Large kernels combine global context too quickly, washing out the local, band-specific details that SubSpectralNet preserves.
  • Sub-classifiers as Regularizers: By forcing the sub-branches to produce their own classifications, the network converges faster and learns more robust features than a standard "black box" CNN.
  • The "Outlier" Effect: Interestingly, for some scenes like "public square," a specific sub-classifier actually performed better than the global classifier. This suggests that the full spectrogram sometimes contains "noise" or outliers that can confuse a global model.

Key Results

  • Accuracy Boost: SubSpectralNet achieved a top accuracy of 74.08% (using 200 mel-bins). This is a significant improvement over the official DCASE 2018 baseline.
  • Efficiency: Even when using only 40 mel-bins, SubSpectralNet reached 72.18% accuracy, outperforming a standard CNN with the same number of parameters (which hit ~66.79%).
  • Convergence: The proposed training method (jointly training sub-classifiers and the global network) resulted in faster convergence during the training phase compared to the baseline.

Limitations and Future Work

  • Manual Parameter Tuning: Currently, the size of the sub-spectrograms and the "hop-size" (overlap) between bands must be set manually. The authors suggest using their statistical analysis to automate this selection in the future.
  • Fusion Logic: Since some sub-classifiers occasionally outperform the global network, there is room to improve how the global network "weights" the information from different bands.
  • Temporal Modeling: The current architecture focuses heavily on frequency bands. The authors plan to explore incorporating temporal modeling (like Recurrent Neural Networks or Transformers) to better capture how these sounds evolve over the 10-second window.

Summary updated

Abstract

Acoustic Scene Classification (ASC) is one of the core research problems in the field of Computational Sound Scene Analysis. In this work, we present SubSpectralNet, a novel model which captures discriminative features by incorporating frequency band-level differences to model soundscapes. Using mel-spectrograms, we propose the idea of using band-wise crops of the input time-frequency representations and train a convolutional neural network (CNN) on the same. We also propose a modification in the training method for more efficient learning of the CNN models. We first give a motivation for using sub-spectrograms by giving intuitive and statistical analyses and finally we develop a sub-spectrogram based CNN architecture for ASC. The system is evaluated on the public ASC development dataset provided for the "Detection and Classification of Acoustic Scenes and Events" (DCASE) 2018 Challenge. Our best model achieves an improvement of +14% in terms of classification accuracy with respect to the DCASE 2018 baseline system. Code and figures are available at https://github.com/ssrp/SubSpectralNet.
BibTeX
@inproceedings{phaye2019subspectralnet,
  author = {Sai Samarth R. Phaye and Emmanouil Benetos and Ye Wang},
  title = {{SubSpectralNet - Using Sub-spectrogram Based Convolutional Neural Networks for Acoustic Scene Classification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  doi = {10.1109/icassp.2019.8683288},
}

Loading related content...

Figure