Reducing Model Complexity for DNN Based Large-Scale Audio Classification

Tan Lee 李丹, Yuzhong Wu, Tan LeeICASSP 2018 · DOI

TL;DR

This paper tackles the challenge of building efficient deep learning models for large-scale audio classification using the massive AudioSet dataset. They demonstrate that while Convolutional Neural Networks (CNNs) offer superior performance, their complexity can be significantly reduced through bottleneck layers and global average pooling, achieving comparable results with a fraction of the parameters.

What Problem Does This Paper Address?

The explosion of online multimedia content has created a pressing need for effective tools to classify, index, and manage audio data. Large-scale audio classification, which involves categorizing audio signals into a vast number of sound categories, is particularly challenging. While deep learning models, especially CNNs, have shown great promise in this domain, their inherent complexity can hinder practical deployment. This paper aims to bridge this gap by investigating methods to reduce the model complexity of DNNs for large-scale audio classification without sacrificing performance.

Methods

The authors explore two primary strategies for reducing model complexity in DNNs for audio classification:

  • Bottleneck Layers: Inspired by their use in speech recognition, bottleneck layers are introduced within fully-connected layers of a neural network. These layers have a significantly smaller number of neurons, forcing the network to learn a compressed, low-dimensional representation of the input features. This effectively acts as an embedding feature extractor. The paper investigates placing these bottleneck layers either at the "final" stage (before the output layer) or "midway" between fully-connected layers.
  • Global Average Pooling: This technique replaces the traditional flattening of convolutional feature maps and subsequent fully-connected layers. Instead, it applies an average pooling operation across each feature map from the last convolutional layer. The resulting pooled features are then directly fed into the classifier. This method drastically reduces the number of parameters by eliminating the need for large fully-connected layers.

Datasets

The primary dataset used in this research is:

  • AudioSet: A large-scale, human-annotated dataset comprising 2 million audio samples from YouTube, covering 527 sound categories. The authors utilize the balanced training and evaluation sets, with a validation set created from 10% of the training data.

The paper also uses:

  • TUT Acoustic Scenes 2016 database: This dataset is used for acoustic scene classification (ASC) to validate the proposed methods on a different, albeit smaller, task.

Key Insights and Ideas

  • CNNs Reign Supreme (for performance): The experiments confirm that CNN-based models generally outperform traditional Multi-Layer Perceptrons (MLPs) and Recurrent Neural Networks (RNNs) for large-scale audio classification tasks.
  • Complexity is the Bottleneck (literally): While CNNs are powerful, their high parameter count is a significant practical hurdle. The paper demonstrates that this complexity can be effectively managed.
  • Bottleneck Layers Offer a Sweet Spot: Placing bottleneck layers, particularly "midway" between fully-connected layers, allows for a substantial reduction in model parameters while maintaining performance close to the original, more complex models.
  • Global Average Pooling is a Game Changer for Efficiency: This technique offers the most dramatic reduction in model size, achieving a reduction of approximately 22 times compared to the original AlexNet(BN) model, with only a slight performance degradation. This makes it highly attractive for deployment on resource-constrained devices.
  • Validation on a Different Task: The successful application of these complexity reduction strategies on the TUT Acoustic Scenes 2016 dataset suggests their generalizability beyond the specific AudioSet task.

Key Results

  • Best Performing Model: The AlexNet with batch normalization (AlexNet(BN)) achieved the highest AUC of 0.927 on the AudioSet evaluation set, outperforming even a 50-layer ResNet.
  • Complexity Reduction with Bottleneck Layers: The "Bneck-Mid-1024" configuration achieved the same AUC as the original AlexNet(BN) with approximately 14% fewer parameters.
  • Maximum Complexity Reduction: The global average pooling strategy reduced the model complexity to 2.59 million parameters, a reduction of about 1/22 compared to the original AlexNet(BN). This model achieved an AUC of 0.916, comparable to the ResNet-50 model.
  • Acoustic Scene Classification: On the TUT Acoustic Scenes 2016 database, the global average pooling strategy resulted in an accuracy of 85.9%, demonstrating its effectiveness in reducing complexity for this task as well.

Limitations and Future Work

  • Limited AudioSet Subset: The study acknowledges that only a small portion of the AudioSet database was used, despite this portion being significantly larger than existing benchmarks. Future work could explore the full dataset.
  • Trade-off Exploration: While the paper explores various configurations, further fine-tuning of bottleneck layer sizes and placement could potentially yield even better trade-offs between performance and complexity.
  • Generalizability: While validated on ASC, further exploration on other audio-related tasks could solidify the general applicability of these complexity reduction techniques.

Summary updated

BibTeX
@inproceedings{2018reducing,
  author = {Tan Lee 李丹 and Yuzhong Wu and Tan Lee},
  title = {{Reducing Model Complexity for DNN Based Large-Scale Audio Classification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2018},
  doi = {10.1109/icassp.2018.8462168},
}
Figure