Introduction
While computer vision has undergone a revolution driven by deep neural networks, acoustic scene classification (ASC)—the task of identifying environments like airports, parks, or streets from audio—has historically lagged behind. Traditional ASC models rarely exceeded ten layers, often due to the lack of massive labeled datasets comparable to ImageNet. This research explores whether architectural innovations from the vision domain, such as residual connections, batch normalization, and global average pooling, can be successfully ported to the audio domain to improve classification robustness.
Methodology
The researchers treated audio classification as an image recognition task by converting signals into log-mel spectrograms. The proposed pipeline involves three critical stages: diverse preprocessing, deep architectural adaptation, and ensembling.
Preprocessing and Architectures
Instead of relying solely on mono audio, the study utilized four distinct spectrogram types: standard Mono, Stereo (L/R), Mid/Side (M/S) (sum and difference of channels), and Harmonic/Percussive (H/P) separation. These inputs were fed into two adapted vision families:
- VGG-inspired models: 8, 10, and 12-layer variants. To improve efficiency, traditional fully connected layers were replaced with Global Average Pooling (GAP), reducing parameters from 138 million to just 5.3 million.
- Residual Networks (ResNet): Standard ResNet-18, 34, and 50 architectures were tested to evaluate the limits of depth and the effectiveness of skip connections in audio.
Key Results and Analysis
The study utilized the DCASE 2018 Challenge Task 1 dataset. A primary finding was the superior performance of Mid/Side (M/S) preprocessing. The "Side" channel (the difference between L and R) appears to effectively isolate spatial information and directional sound events that are often obscured in a mono mix.
As shown in the results table, the VGG-10 model using M/S preprocessing achieved the highest single-model accuracy of 77.7%. Interestingly, deeper was not always better; the ResNet-50 model showed signs of severe overfitting, likely because the dataset size was insufficient for its high capacity. By ensembling 24 different model/preprocessing combinations (and excluding the overfitted ResNet-50), the final accuracy reached 79.3%, outperforming the DCASE 2018 baseline by nearly 20 percentage points.
Discussion and Limitations
Several key insights emerged regarding model regularization and transfer learning:
- Regularization: The inclusion of Batch Normalization was critical, yielding up to a 10% improvement in accuracy by stabilizing internal activations.
- Transfer Learning: Attempts to use weights pre-trained on ImageNet did not match the performance of models trained from scratch on audio. This suggests that while vision architectures translate well, the specific features of natural images (edges, textures) differ significantly from the patterns in spectrograms.
- Semantic Confusion: Despite high overall accuracy, models still struggle with semantically similar environments. Confusion matrices revealed significant overlap between "airport" and "shopping mall," as well as "tram" and "bus," which share similar acoustic signatures.
Future work will likely require larger datasets to unlock the potential of even deeper architectures and further exploration into raw audio representations, which currently remain more difficult to train than spectral inputs.
Summary updated