Efficient Robust Music Genre Classification with Depthwise Separable Convolutions and Source Separation

Gabriel MersyAAAI 2021 · DOI

Abstract

Music genre classification remains a challenging task in Music Information Retrieval (MIR), particularly when distinguishing between acoustically similar sub-genres. This paper explores a novel approach that leverages deep source separation to decompose audio into constituent instrument tracks—bass, drums, and others—which are then utilized as multi-channel inputs for convolutional neural networks (CNNs). Our results demonstrate that this "unmixing" process significantly improves classification accuracy and robustness compared to standard single-spectrogram methods. Furthermore, we evaluate the efficiency of depthwise separable convolutions (DS-CNNs), finding them to be a viable, low-parameter alternative for resource-constrained environments, albeit with increased sensitivity to data scarcity.

Introduction

Standard approaches to music genre classification typically rely on a single mel-spectrogram representing the entire audio track. However, this "single-image" representation often obscures fine-grained features, such as specific instrumental rhythms or timbres, because all signals are layered together. This is particularly problematic for genres like Electronic Dance Music (EDM), where stylistic differences can be subtle. This research investigates whether pre-processing audio with source separation can provide clearer, more discriminative features for a learner by treating different instrument stems as distinct input channels.

Methodology

Source Separation and Feature Engineering

The proposed pipeline utilizes a pre-trained U-net model (Spleeter) to decompose audio into four stems: bass, drums, vocals, and "other." By treating these stems as analogous to the RGB channels of a color image, the author creates a 3-channel feature tensor (bass, drums, and other). This allows the CNN to learn spatial and temporal relationships between specific instrument groups that would otherwise be merged in a grayscale-like single spectrogram.

Architectural Comparisons

The study compares two primary CNN architectures:

  1. Standard 2D Convolution: Utilizes traditional 3D filters to process the multi-channel input.
  2. Depthwise Separable Convolution (DS-CNN): An efficient alternative that applies a single filter to each channel independently before combining them. This architecture drastically reduces the parameter count (e.g., from 9,000 to approximately 3,000 for a $3 \times 3$ kernel), making it suitable for mobile deployment.

Experimental Setup

The models were evaluated on a specialized EDM dataset comprising 66 songs from the GiantSteps+ EDM Key dataset and the Reddit Free Download Collection, categorized into six genres. To ensure statistical validity, models were trained for six epochs across 50 independent trials. The audio was segmented into 10-second clips at a sample rate of 22,050 Hz.

Results

The experiments revealed a significant performance boost when using source-separated inputs. The 3-channel standard 2D convolution model achieved the highest performance, outperforming the single-spectrogram baseline by a wide margin.

Table 1: Performance comparison of single-spectrogram vs. multi-channel source-separated models.

As shown in Table 1, the 3-spec 2DConv model reached an accuracy of 0.456 and an F1 score of 0.435, compared to the baseline's 0.384 accuracy and 0.371 F1. Notably, the depthwise separable model (3-spec, DW-2DConv) maintained competitive performance (0.451 accuracy) while utilizing far fewer parameters.

Discussion and Conclusion

While source separation acts as a powerful non-linear feature extractor, the choice of architecture involves a trade-off between efficiency and stability.

Figure 1: F1 score distributions across 50 trials comparing standard and depthwise separable convolutions.

Analysis of the F1 score distributions (Figure 1) indicates that while DS-CNNs are computationally efficient, they exhibit higher instability—characterized by a left-skewed distribution—compared to standard convolutions. This suggests that in low-data regimes, extreme parameter reduction may make models more sensitive to initialization and specific samples.

In conclusion, source separation provides a clear advantage for classifying complex, instrumentally dense genres like EDM. Future work will focus on scaling this approach to larger datasets and refining the regularization of DS-CNNs to mitigate instability in small-data scenarios.

Summary updated

BibTeX
@inproceedings{mersy2021efficient,
  author = {Gabriel Mersy},
  title = {{Efficient Robust Music Genre Classification with Depthwise Separable Convolutions and Source Separation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17982},
}
Figure