Acoustic Scene Classification with Mismatched Devices Using CliqueNets and Mixup Data Augmentation

Truc Nguyen, Franz PernkopfInterspeech 2019

TL;DR

This paper tackles the challenge of acoustic scene classification (ASC) when the recording devices used during testing differ from those used during training. They propose an efficient deep learning model combining "CliqueNets" for feature learning and a "mixture-of-experts" (MoE) layer for classification, along with "mixup" data augmentation to balance imbalanced datasets. This approach achieves competitive performance with significantly fewer parameters than existing methods.

What Problem Does This Paper Address?

Acoustic Scene Classification (ASC) aims to identify the environment a sound is recorded in (e.g., street, metro, park). While deep learning, particularly Convolutional Neural Networks (CNNs), has significantly boosted ASC performance, many state-of-the-art models are computationally expensive, requiring many layers, large ensembles, or complex architectures. A critical real-world challenge is the "mismatched devices" problem: models trained on data from one set of recording devices often perform poorly when tested on data from different devices due to variations in audio capture. This is particularly problematic in datasets like DCASE 2018 Task 1B, where data from different devices is imbalanced, leading to overfitting on the dominant device's data. The authors aim to develop a robust and resource-efficient ASC model that can handle this device mismatch and data imbalance.

Methods

The proposed system integrates several key components:

  • CliqueNets: This is a novel CNN architecture designed for efficient feature learning. Unlike standard CNNs, CliqueNets employ a recurrent feedback structure within "clique blocks." This allows for iterative feature refinement by propagating information both forward and backward between layers within a block. This iterative process helps maximize information flow and learn richer features without drastically increasing model complexity. The authors specifically use a simplified version with two clique blocks.
  • Mixture-of-Experts (MoE) Layer: To enhance classification robustness, the authors replace the traditional fully connected output layer with a dense MoE layer. This layer consists of multiple "expert" networks, each specializing in different aspects of the input features. A "gating network" then learns to dynamically select and combine the outputs of these experts for the final classification. This allows the model to leverage diverse feature representations.
  • Mixup Data Augmentation: To address the data imbalance issue (where Device A has significantly more data than Devices B and C), the authors employ "mixup." Mixup creates synthetic training examples by taking convex combinations of existing data samples and their corresponding labels. This technique is used to artificially increase the data from the underrepresented devices (B and C), balancing the dataset and preventing overfitting. It's also used as a general data augmentation strategy on the entire balanced training set to further improve performance.

Datasets

The primary dataset used is the DCASE 2018 Task 1B dataset. This dataset contains recordings of 10 urban acoustic scenes from six European cities, recorded using different devices. The authors specifically focus on the challenge posed by the imbalanced distribution of data across these devices (Device A having substantially more data than Devices B and C). They use a portion of the development set for training and evaluation.

Key Insights and Ideas

  • Resource Efficiency is Key: The paper strongly emphasizes that achieving high performance in ASC doesn't necessarily require massive models. Their CliqueNet architecture, designed for efficient feature refinement, demonstrates this.
  • Iterative Refinement Works: The recurrent feedback mechanism in CliqueNets allows for a more thorough processing of features, leading to better representations compared to purely feed-forward architectures, especially when dealing with complex acoustic scenes.
  • MoE for Robustness: Using a mixture of experts proves effective in handling the diverse and potentially noisy features extracted from different recording devices, leading to more generalized classification.
  • Mixup is a Powerful Tool for Imbalance: The application of mixup is crucial for mitigating the negative impact of imbalanced datasets in ASC, significantly improving performance on underrepresented devices.

Key Results

The proposed model, a CliqueNet with 2 blocks and 10 MoE experts (Cli_2_36_6_10E_muBC), achieved:

  • An average classification accuracy of 64.7% for Devices B and C, and 70.0% for Device A.
  • This performance was achieved with less than one million parameters, significantly reducing model complexity compared to other state-of-the-art models.
  • The mixup data augmentation strategy led to significant performance improvements for Devices B and C (up to 6.7% and 5.0% absolute improvement, respectively) compared to the same model without balancing.
  • The CliqueNet architecture, even with only Stage-I processing (similar to DenseNets), outperformed deeper DenseNet structures in terms of accuracy and parameter count.

Limitations and Future Work

  • Metro Scene Difficulty: The paper notes that the "metro" scene remains the most challenging for their model, although performance is still slightly better than previous work.
  • Further Optimization of MoE: While 10 experts showed optimal performance, the paper doesn't extensively explore the impact of even more experts or different gating mechanisms.
  • Generalizability to Other Datasets: While effective on DCASE 2018 Task 1B, further validation on other mismatched device datasets would be beneficial.

The authors suggest that their approach of combining efficient feature learning (CliqueNets) with robust classification (MoE) and effective data augmentation (mixup) provides a strong foundation for future research in resource-constrained ASC systems.

Summary updated

BibTeX
@inproceedings{nguyen2019acoustic,
  author = {Truc Nguyen and Franz Pernkopf},
  title = {{Acoustic Scene Classification with Mismatched Devices Using CliqueNets and Mixup Data Augmentation}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure