On the Importance of Audio-Source Separation for Singer Identification in Polyphonic Music

Bidisha Sharma, Rohan Kumar Das, Haizhou Li

Abstract

Identifying singers in studio recordings is a complex task due to the constant presence of background accompaniment. This paper demonstrates that background "noise" is the primary bottleneck for singer identification in polyphonic music. By integrating Wave-U-Net for audio-source separation into a standard i-vector identification pipeline, the authors achieve a significant boost in accuracy, particularly in data-scarce scenarios.

Introduction

Identifying a singer in a polyphonic mix is significantly more difficult than speaker identification in clean audio. In music, the target signal (the voice) is overlaid with loud, non-stationary interference from instruments like drums, bass, and guitars, which often share overlapping frequency ranges. While previous methods attempted to ignore or simply filter this background, this study evaluates the impact of state-of-the-art audio-source separation on extracting cleaner, singer-specific timbral features.

Proposed Methodology

The researchers proposed a multi-stage pipeline designed to isolate the vocal signal and extract robust identity features.

The proposed singer identification pipeline, featuring audio-source separation, feature extraction, and an i-vector modeling system.
  1. Audio-Source Separation: Three methods were compared: traditional Harmonic/Percussive separation, a CNN-based approach, and Wave-U-Net. Wave-U-Net, a 1D time-domain neural network, was selected for the final system as it preserves fine details better than spectrogram-based models.
  2. Vocal/Non-vocal Segmentation: An energy-based Voice Activity Detection (VAD) step filters out remaining instrumental-only segments.
  3. Feature Extraction: 60-dimensional features (20 MFCCs plus derivatives) are extracted from the separated vocals.
  4. Singer Modeling: The i-vector framework is used to compress acoustic data into low-dimensional identity vectors. Linear Discriminant Analysis (LDA) is then applied to maximize the separability between different singers.

Experimental Setup

The system was evaluated using the Artist20 dataset, containing 1,413 tracks from 20 artists. A 6-fold cross-validation was performed, training on five albums and testing on the sixth to ensure the model learns singer identity rather than album-specific production qualities. Wave-U-Net was pre-trained on the MUSDB18 dataset.

Results and Discussion

The inclusion of source separation proved to be a "force multiplier" for identification accuracy.

Accuracy comparison between the baseline system and the proposed framework across 6-fold cross-validation.

Performance Gains

The proposed framework achieved an average accuracy of 89.97%, a 5.66% absolute improvement over the baseline (84.31%) that did not use source separation. Notably, Wave-U-Net (90.41% on a single fold) significantly outperformed CNN-based separation (78.54%), which introduced artifacts that hindered identification.

Robustness in Short Segments

The most significant impact was observed in the "short clip" scenario. When tested on only 10 seconds of audio, the proposed system maintained 68.04% accuracy, whereas the baseline dropped to 47.03%. This ~21% improvement highlights that removing musical "clutter" is essential when temporal context is limited.

Conclusion

This research confirms that background music actively distorts the features defining a singer's identity. By using advanced source separation like Wave-U-Net, systems can extract much cleaner features, leading to more robust identification even with very short audio samples. Future work will focus on refining the VAD process to handle residual artifacts and extending the system to open-set singer verification.

Summary updated

Abstract

Singer identification is to automatically identify the singer in a music recording, such as a polyphonic song. A song has two major acoustic components that are singing vocals and background accompaniment. Although identifying singers is similar to speaker identification, it is challenging due to the interference of background accompaniment on the singer-specific information in singing vocals. We believe that separating the background accompaniment from the singing vocal will help us to overcome the interference. In this work, we extract the singing vocals from polyphonic songs using Wave-U-Net based audio-source separation approach. The extracted singing vocals are then used in i-vector based singer identification system. Further, we explore different state-of-the-art audio-source separation methods to establish the role of considered method in application to singer identification. The proposed singer identification framework achieves an absolute accuracy improvement of 5.66% over the baseline without audio-source separation.
BibTeX
@inproceedings{sharma2019importance,
  author = {Bidisha Sharma and Rohan Kumar Das and Haizhou Li},
  title = {{On the Importance of Audio-Source Separation for Singer Identification in Polyphonic Music}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-1925},
}
Figure