Source Separation and Depthwise Separable Convolutions for Computer Audition (Student Abstract)

Gabriel Mersy, Jin Hong KuanAAAI 2021 · DOI

Abstract

Traditional computer audition systems often struggle to distinguish between acoustically similar music genres when relying on single-spectrogram representations. This study investigates whether source separation—decomposing audio into individual instrumental components—can improve classification performance in data-limited scenarios. By treating separated tracks (bass, drums, and others) as distinct input channels for a convolutional neural network (CNN), we demonstrate a significant improvement in F1 scores for Electronic Dance Music (EDM) genre recognition. Furthermore, we evaluate the trade-offs between standard 2D convolutions and more efficient depthwise separable convolutions.

Introduction

Most machine listening systems utilize a "macro-scale" representation, such as a single mel spectrogram, which merges all instrumental layers into one. While effective for broad genre classification, this approach often fails to capture the "micro-scale" nuances—specific timbres or rhythmic patterns—necessary to differentiate closely related genres. For instance, preliminary tests showed standard CNNs achieving only 27% accuracy when distinguishing rock from country. This research explores whether providing models with "decoded" instrumental components, mimicking human auditory perception, can enhance feature extraction from small datasets.

Methodology

The proposed pipeline transforms audio into a multi-channel feature tensor through the following steps:

  1. Source Separation: Stereo audio is decomposed into four tracks (bass, drums, vocals, and "other") using a pre-trained Spleeter U-net model.
  2. Feature Representation: Mono mel spectrograms are generated for the bass, drums, and "other" tracks. These are stacked to form a 3-channel tensor, analogous to an RGB image.
  3. Model Architectures: We compared a standard 2D CNN (multichannel convolution) against a Depthwise Separable CNN. The latter processes channels independently before combining them, aiming to reduce parameters and mitigate overfitting.
  4. Experimental Setup: Models were evaluated on a specialized EDM dataset comprising 66 songs across six genres. Training involved 50 independent trials with 80/20 train-test splits to ensure statistical robustness.

Results

The source-separated approach clearly outperformed the traditional single-spectrogram baseline.

Comparison of Accuracy and F1 scores across different model configurations.

As summarized in the table above, the 3-spectrogram 2D CNN was the top performer, achieving an accuracy of 0.456 and an F1 score of 0.435. This represents a statistically significant improvement over the baseline (p-value = 0.04). Interestingly, the inclusion of vocals had little effect on classifier performance for these EDM genres, suggesting the genre identity was primarily driven by the rhythm and synth sections.

F1 score distributions over 50 trials for various model comparisons.

The distribution of F1 scores across 50 trials highlights the performance gains of the 3-spectrogram models. However, it also reveals that while depthwise separable convolutions are designed for efficiency, they exhibited more instability—indicated by a left-skewed distribution—compared to the standard 2D CNN. This suggests that the reduced parameter count may be insufficient for reliable convergence on such a small dataset.

Discussion and Conclusion

This work demonstrates that source separation provides a clear advantage for music genre classification in limited-data environments. By isolating instrumental tracks, CNNs can more effectively identify salient genre-specific features. While depthwise separable convolutions offer efficiency, their observed instability in this context suggests a need for further investigation into regularization or larger datasets. Future research will explore the scalability of this approach to larger libraries and the impact of source separation artifacts on classification accuracy.

Summary updated

BibTeX
@inproceedings{mersy2021source,
  author = {Gabriel Mersy and Jin Hong Kuan},
  title = {{Source Separation and Depthwise Separable Convolutions for Computer Audition (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17920},
}
Figure