Meta learning based audio tagging

Kele Xu, Boqing Zhu, Dezhi Wang, Yuxing Peng, Huaimin Wang, Lilun Zhang, Bo LiDCASE 2018 · OpenReview

Abstract

Audio tagging—the task of identifying specific sound events within an acoustic scene—presents significant challenges due to overlapping events, varying durations, and background noise. This paper describes a high-performing solution for the DCASE 2018 challenge that leverages a two-level meta-learning architecture. By integrating diverse deep convolutional neural networks (CNNs) with handcrafted statistical features via stacked generalization, the system achieves a mean Average Precision at 3 (mAP@3) of 0.977, far exceeding the competition baseline of 0.704.

Introduction

While computer vision often dominates the machine learning spotlight, audio tagging remains a complex and distinct domain. The primary difficulties include data scarcity, label noise in human-provided annotations, and the challenge of effectively fusing "shallow" handcrafted features with "deep" neural network representations. This research addresses these gaps by developing a robust, general-purpose system capable of categorizing audio clips into 41 distinct classes from the AudioSet ontology.

Methodology

The proposed system utilizes a two-level hierarchical architecture designed to maximize feature diversity and model robustness.

Level 1: Diverse Single Models

The first level consists of multiple deep CNNs trained on various architectures (Xception, ResNet, ResNeXt, and Dual Path Networks) and input types (raw waveforms, log-Mel spectrograms, and MFCCs). To enhance spectral inputs, temporal derivatives (delta and delta-delta) were included to create three-channel inputs. A critical component of the training process was the use of mixup data augmentation, which linearly combines random training samples and their labels to encourage the model to learn more generalized class boundaries.

Level 2: Meta-Learning Ensemble

The second level employs Stacked Generalization to fuse the outputs of the Level 1 models. Instead of simple averaging, the predicted probabilities from five distinct CNNs are concatenated to form "meta-features." These are then augmented with handcrafted statistical features, including the maximum, minimum, variance, and skewness of the MFCCs.

The Level 2 meta-learning architecture using stacked generalization, combining CNN outputs with statistical features for XGBoost classification.

A Gradient Boosting Machine (XGBoost) is trained on this combined feature set. This meta-learner identifies which base models are most reliable for specific sound categories, effectively learning an optimal fusion strategy.

Results and Discussion

The experimental results demonstrate the efficacy of the stacking approach and the importance of model diversity.

Performance comparison of various single model architectures and configurations on the DCASE 2018 dataset.

Key findings from the evaluation include:

  • Impact of Mixup: Mixup augmentation consistently improved performance across all architectures, proving vital for generalization on limited datasets.
  • Transfer Learning: Initializing weights from models pre-trained on ImageNet significantly boosted results for spectrogram-based inputs, demonstrating successful cross-domain knowledge transfer.
  • Feature Complementarity: The highest performance was achieved by combining models that processed different input representations (e.g., raw waves vs. spectrograms).
  • Hybrid Robustness: The inclusion of "shallow" statistical features alongside "deep" CNN features provided additional robustness that deep learning alone could not capture.

The final ensemble reached a mAP@3 of 0.977 on the public leaderboard and 0.951 on the private leaderboard, representing a substantial improvement over the baseline.

Conclusion

This meta-learning framework provides a powerful solution for audio tagging by intelligently combining diverse deep learning models and traditional statistical features. While the current system uses 1.5-second audio snippets, future work will explore longer temporal contexts and evaluate the framework's scalability on larger datasets like the full Google AudioSet.

Summary updated

BibTeX
@inproceedings{xu2018meta,
  author = {Kele Xu and Boqing Zhu and Dezhi Wang and Yuxing Peng and Huaimin Wang and Lilun Zhang and Bo Li},
  title = {{Meta learning based audio tagging}},
  booktitle = {DCASE},
  year = {2018},
  url = {https://openreview.net/forum?id=UItFYy5qMt},
}
Figure