TL;DR
Researchers have developed a way to make singer identification (SID) models more robust by breaking the "bond" between a singer's voice and their typical musical style. By using source separation to "shuffle-and-remix" vocals with different instrumental tracks and incorporating melodic contours, they achieved a significant jump in identification accuracy on the benchmark artist20 dataset.
What Problem Does This Paper Address?
Identifying a singer in a recording (Singer Identification, or SID) seems straightforward, but background music creates a massive "confounding" problem. Most singers have a preferred genre or production style. If a model is trained on standard studio mixes, it might accidentally learn to identify the instruments or the genre (e.g., "this sounds like a jazz arrangement") rather than the actual vocal characteristics of the artist.
This leads to poor generalization: a model trained this way might fail to recognize a singer if they perform an acoustic version of a song or switch genres. The authors aim to force the model to focus strictly on the voice by decoupling it from the accompaniment.
Methods
The authors propose a two-pronged approach involving a novel model architecture and a clever data augmentation strategy:
- Source Separation: They use Open-Unmix, a state-of-the-art tool, to split songs into separate vocal and instrumental tracks.
- Shuffle-and-Remix Augmentation: Instead of just training on the original songs, they create a synthetic dataset. They take the vocal track from Singer A and mix it with the instrumental track from a completely different song by Singer B. This forces the model to recognize Singer A regardless of the musical context.
- CRNNM Architecture: They build upon a standard Convolutional Recurrent Neural Network (CRNN). Their version, called CRNNM, adds a secondary branch that processes melodic features (extracted using the CREPE pitch tracker). This branch helps the model capture the "timbre" and "style" of the melody contour alongside the raw spectral data from the mel-spectrogram.
- Training Logic: The model processes 5-second audio segments. For the final "per-song" prediction, it uses a majority vote across all segments in that song.
Datasets
The study uses the artist20 dataset, a standard benchmark in the field.
- Composition: 20 artists, each with 6 albums, totaling 1,413 tracks.
- Evaluation Split: The authors use an album-split (training on some albums and testing on others). This is much harder than a "song-split" because it prevents the model from "cheating" by memorizing the specific production quality or "vibe" of a particular recording session.
Key Insights and Ideas
- Breaking the "Bonds": The most important insight is that simply removing the accompaniment (Vocal-only training) isn't the best solution. In fact, training on vocals alone performed worse than the original mixes in some tests. The "shuffle-and-remix" method is superior because it teaches the model what is invariant about a voice across many different backgrounds.
- Melody as Timbre: While pitch is usually seen as "what" is being sung, the way a singer moves between pitches (the contour) carries information about their vocal technique and identity.
- The "Vocalness" Correlation: The researchers found a correlation (0.39) between the volume of the vocal track and the model's confidence. If a 5-second segment has very quiet vocals or just an instrumental break, the model's accuracy plummets. This highlights that SID models are highly sensitive to the signal-to-noise ratio of the voice.
Key Results
The combination of the new architecture and data augmentation yielded impressive gains:
- Baseline (CRNN on original data): Achieved a song-level F1 score of 0.67.
- Proposed (CRNNM + Data Augmentation): Achieved a song-level F1 score of 0.75.
- Visualization: t-SNE plots (which project complex data into 2D space) showed that the embeddings from the CRNNM model were much more clearly clustered by artist than the baseline model, proving the features learned were more discriminative.
Limitations and Future Work
- Silence and Instrumental Gaps: The model currently struggles with segments where the singer isn't active. The authors suggest adding a "vocal detector" as a pre-filter to skip non-vocal segments during inference.
- Musical Coherence: The current "shuffle-and-remix" method is musically "blind"—it might mix a vocal in C-major with an instrumental in F-minor at a different tempo. Future work could involve "key-aware" and "tempo-aware" remixing to create more natural-sounding training data.
- Architecture Refinement: The authors plan to experiment with replacing the convolutional layers in the melody branch with more GRUs, as melodic contours are essentially time-series data.
Summary updated