A Hybrid of Deep Audio Feature and i-vector for Artist Recognition

Jiyoung Park, Donghyun Kim, Jongpil Lee, Sangeun Kum, Juhan NamarXiv 2018 · arXiv

Introduction

Identifying a musical artist from a short audio snippet is a complex task. Unlike speech recognition, where the target is a specific vocal tract, "artist recognition" involves abstract stylistic elements including timbre, harmony, rhythm, and vocal characteristics. Traditional methods have relied on hand-crafted features or generative models like i-vectors, while modern approaches favor discriminative Deep Convolutional Neural Networks (DCNNs). This research investigates a hybrid approach that combines the strengths of both paradigms to achieve state-of-the-art performance.

Methodology

The study evaluates two primary architectures and their combinations:

  1. i-vector (Generative Path): Leveraging techniques from speaker verification, this model uses Mel-Frequency Cepstrum Coefficients (MFCCs) and Gaussian Mixture Models (GMMs) to create a statistical "fingerprint" of spectral textures. It is particularly robust at capturing vocal characteristics.
  2. DCNN (Discriminative Path): A five-layer convolutional network processes 3-second mel-spectrograms. Instead of using the final classification, the 256-dimensional vector from the last hidden layer is extracted as a "deep audio feature."
  3. Fusion Strategies: The models were combined via Early Fusion (concatenating features before scoring) and Late Fusion (averaging independent model scores). Probabilistic Linear Discriminant Analysis (PLDA) served as the backend scoring engine for both.
Similarity matrices for i-vector (top) and DCNN (bottom) models trained on varying numbers of artists (100, 1,000, and 2,000).

Experimental Results

The models were trained and tested using the Million Song Dataset (MSD), specifically focusing on 500 "unseen" artists for evaluation to ensure the system learned generalizable stylistic features rather than specific recording artifacts.

A key finding was the differing scalability of the two approaches. As shown in the performance metrics, the DCNN's accuracy continues to climb as the number of training artists increases. In contrast, the i-vector model's performance tends to plateau, suggesting it is more robust with limited data but less capable of capturing the nuances of a massive, diverse artist space.

Performance comparison (EER and Accuracy) across different training set sizes for i-vector, DCNN, and fusion models.

The Superiority of Late Fusion

The results demonstrate that Late Fusion consistently provides the best performance across all metrics. While Early Fusion surprisingly degraded performance—likely due to raw feature interference—Late Fusion successfully leveraged the complementary nature of generative spectral fingerprints and discriminative deep features.

Task i-vector DCNN Late Fusion (Hybrid)
Artist Verification (EER %) 10.785 8.938 7.813
Artist Identification (Acc.) 0.464 0.408 0.480
Singer Identification (Acc.) 0.560 0.435 0.760

The hybrid approach was most effective for Singer Identification, reaching an accuracy of 0.760. This highlights the i-vector's strength in vocal tract modeling combined with the DCNN's ability to model broader musical context.

Conclusion

Combining traditional generative i-vectors with modern discriminative DCNNs via late fusion yields a superior system for artist recognition. While the DCNN excels at learning from large-scale data, the i-vector remains a powerful tool for capturing unique vocal characteristics. Future work will focus on optimizing the computational efficiency of this dual-pipeline approach and refining artist profile aggregation for music recommendation engines.

Summary updated

Abstract

Artist recognition is a task of modeling the artist's musical style. This problem is challenging because there is no clear standard. We propose a hybrid method of the generative model i-vector and the discriminative model deep convolutional neural network. We show that this approach achieves state-of-the-art performance by complementing each other. In addition, we briefly explain the advantages and disadvantages of each approach.
BibTeX
@misc{park2018hybrid,
  author = {Jiyoung Park and Donghyun Kim and Jongpil Lee and Sangeun Kum and Juhan Nam},
  title = {{A Hybrid of Deep Audio Feature and i-vector for Artist Recognition}},
  year = {2018},
  eprint = {1807.09208},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1807.09208},
}
Figure