DNN Transfer Learning Based Non-Linear Feature Extraction for Acoustic Event Classification

Seongkyu Mun, Minkyu Shin, Suwon Shon, Wooil Kim, David K. Han, Hanseok Ko

TL;DR

This paper introduces a novel feature extraction method for Acoustic Event Classification (AEC) that leverages Deep Neural Networks (DNNs) and transfer learning. By pre-training a DNN on large-scale general audio datasets and then adapting it to specific target environments, the authors create a "non-linear filter" that significantly outperforms traditional linear features (like MFCCs or NMF) and shows remarkable robustness in noisy surveillance settings.

What Problem Does This Paper Address?

Acoustic Event Classification—the ability for a system to autonomously recognize sounds like breaking glass, footsteps, or sirens—faces two major hurdles:

  1. Data Scarcity: Unlike speech recognition or computer vision, AEC lacks massive, high-quality labeled datasets. Most target databases are small and "weakly labeled," making it hard to train complex models from scratch.
  2. Linearity Constraints: Traditional acoustic features (MFCCs, PLP) or learned filters (NMF, K-SVD) are largely based on linear combinations or human auditory models. These often fail to capture the complex, non-linear relationships between different frequency bands that characterize real-world sounds.

The authors aim to bridge this gap by using transfer learning to "borrow" knowledge from large, diverse audio sets and apply it to specific, data-poor tasks.

Methods

The core innovation is the DNN-based non-linear feature extraction scheme. Instead of using a DNN solely as a classifier, the authors use it as a sophisticated, learned filter bank.

  • Source Domain Pre-training: A DNN (3 hidden layers) is first trained on a massive "source" dataset containing 93 different sound classes.
  • Transfer and Adaptation: The output layer of the pre-trained network is removed. Two new adaptation layers and a new output layer are added. The network is then fine-tuned on the "target" domain (e.g., indoor surveillance sounds).
  • Feature Extraction: Once trained, the output layer and the final activation functions are stripped away. The remaining five hidden layers act as the "DNN filter." The raw values from the last hidden layer (a "bottleneck" approach) are extracted as the new acoustic features.
  • Back-end Classification: These extracted features are then processed via standard techniques like Discrete Cosine Transform (DCT) or Principal Component Analysis (PCA) for dimensionality reduction and fed into a final classifier (SVM, GMM, or another DNN).

Datasets

The study utilizes a two-stage data approach:

  • Source Domain (The "Teacher"): A merged dataset of 93 classes from four major libraries: CLEAR-OL, RWCP, UrbanSound, and ESC-50. This provides a broad "background" of acoustic knowledge.
  • Target Domain (The "Student"): An indoor surveillance database consisting of 15 event types (e.g., crying child, gunshot, explosion).
  • Noise Evaluation: To test real-world robustness, the authors added background noise from the ETSI database at varying Signal-to-Noise Ratios (5, 10, and 15 dB).

Key Insights and Ideas

  • Non-Linearity is King: By using non-linear activation functions (Sigmoid) within the DNN filter, the system can model complex inter-frequency relationships that linear methods like NMF simply miss.
  • Transfer Learning Works for Audio: The paper demonstrates that parameters learned from general environmental sounds (like "drilling" or "dog barking") provide a powerful initialization for detecting specific surveillance events (like "breaking glass").
  • Features vs. Classifiers: A key "aha" moment is that the DNN's internal representations (the hidden layers) are often more valuable than its final classification output. Using the DNN as a feature extractor allows researchers to pair it with robust back-end classifiers like SVMs, which can perform better on small datasets.

Key Results

The proposed method showed a clear advantage over both traditional and other deep-learning-based features:

  • Superior Accuracy: The proposed "DNN filter + SVM + DCT" setup achieved an average accuracy of 95.8% across clean and noisy conditions.
  • Outperforming Baselines: It significantly beat standard MFCCs (88.5%), NMF (91.9%), and K-SVD (92.5%).
  • Noise Robustness: In high-noise environments (5 dB SNR), the proposed method maintained high performance (92.5% in living room noise), whereas MFCCs dropped to 79.7%.
  • Transfer Learning Gain: Comparing a model trained from scratch (Case B) to the transferred model (Case C) showed a jump from 93.0% to 95.8%, proving the effectiveness of the source-domain pre-training.

Limitations and Future Work

While the results are impressive, the authors note a few areas for further exploration:

  • Input Feature Sensitivity: The study found that simple DFT (Discrete Fourier Transform) magnitudes worked better as inputs to the DNN filter than raw waveforms or complex (real/imaginary) DFT values. They suggest that raw waveforms might require much larger, more complex networks to be effective.
  • Generalization: The current work focuses on indoor surveillance. Future research will investigate how this transfer learning scheme holds up across other diverse acoustic domains and how different data configurations affect performance.

Summary updated

Abstract

Recent acoustic event classification research has focused on training suitable filters to represent acoustic events. However, due to limited availability of target event databases and linearity of conventional filters, there is still room for improving performance. By exploiting the non-linear modeling of deep neural networks (DNNs) and their ability to learn beyond pre-trained environments, this letter proposes a DNN-based feature extraction scheme for the classification of acoustic events. The effectiveness and robustness to noise of the proposed method are demonstrated using a database of indoor surveillance environments.
BibTeX
@article{mun2017dnn,
  author = {Seongkyu Mun and Minkyu Shin and Suwon Shon and Wooil Kim and David K. Han and Hanseok Ko},
  title = {{DNN Transfer Learning Based Non-Linear Feature Extraction for Acoustic Event Classification}},
  journal = {IEICE Trans. Information and Systems},
  year = {2017},
  url = {https://openreview.net/forum?id=-xJnPskED0},
}

Loading related content...

Figure