This paper, presented at AAAI-2021, introduces a novel attention mechanism designed to capture the subtle, complex nuances required for Singer Identification (SID). By extending the popular "Non-Local" (NL) operation to look across different channels and layers, the authors achieve state-of-the-art results on benchmark datasets.
TL;DR
The authors propose the Fully Generalized Non-Local (FGNL) module, which improves deep learning models by capturing long-range dependencies not just across time and space, but also across different feature channels and network layers. When applied to Singer Identification, this approach significantly outperforms standard attention mechanisms by better isolating the "fine-grained" vocal characteristics of individual artists.
What Problem Does This Paper Address?
Identifying a singer in a recording is a "fine-grained" classification task. Unlike identifying a broad category (like "dog" vs. "car"), the differences between two professional singers can be incredibly subtle. Furthermore, background instruments often mask these vocal cues.
Existing deep learning models often use Non-Local (NL) operations (a form of self-attention) to capture long-range relationships in audio. However, standard NL operations have a blind spot: they typically only look at correlations within a single feature map or across a collapsed set of channels. They ignore the rich interactions between different types of features (channels) and different levels of abstraction (layers).
The authors argue that a singer's unique "voice print" is hidden in these cross-channel and cross-layer correlations, which standard models fail to capture.
Methods
The core contribution is the FGNL module, which is designed as a "plug-and-play" component for existing architectures like Convolutional Recurrent Neural Networks (CRNNs).
As shown in the architecture diagram above, the module consists of three main components:
- The FGNL Operation: Unlike standard NL, which computes responses by attending to positions within the same layer, FGNL concatenates feature maps from multiple layers. It uses a "rolling function" to shift elements along the channel axis, allowing the model to explicitly learn correlations between different feature channels and hierarchical layers.
- Gaussian Smoothing: To prevent the model from over-focusing on "noise" within the high-dimensional feature maps, the authors integrate a depth-wise convolution with a Gaussian kernel. This smooths the feature maps before the attention weights are calculated, leading to better generalization.
- Modified Squeeze-and-Excitation (MoSE): Standard SE blocks use fully connected layers to recalibrate channel weights. The authors modify this into a convolutional scheme (MoSE) at the end of the FGNL module to adaptively emphasize the most relevant correlated features for identifying the target singer.
Datasets and Evaluation
The researchers evaluated their model on the artist20 dataset, a standard benchmark for SID containing 1,413 songs from 20 different artists. They employed an "album-split" protocol to ensure the model identifies the singer's voice rather than memorizing specific production styles from a single album. They tested both original audio and "vocal-only" versions separated via the Open-Unmix tool.
Key Insights and Results
- Beyond Spatio-Temporal: The most significant insight is that "non-local" should mean more than just "far away in time or space." In a deep neural network, a "position" is defined by its coordinates, its channel, and its layer depth. FGNL is "fully generalized" because it looks across all these dimensions.
- Smoothing for Fine-Grained Tasks: In tasks where subtle differences matter, high-frequency noise in feature maps can lead to overfitting. The inclusion of Gaussian smoothing acts as a regularizer that helps the attention mechanism focus on broader, more reliable vocal patterns.
- State-of-the-Art Performance: The FGNL module provided a substantial boost across all testing scenarios. On the
artist20dataset, the baseline CRNN achieved a "Best F1" score of 0.53 (10s clips), while adding the FGNL module boosted this to 0.58. When combined with a melody-aware architecture (CRNNM), the FGNL version reached an F1 score of 0.83, setting a new high bar for the dataset. - Efficiency: Interestingly, the
CRNN_FGNLmodel achieved better performance than the more complexCRNNMwhile using fewer parameters, suggesting that sophisticated attention is more effective than simply adding specialized branches.
Limitations and Future Work
While the results are impressive, the authors note that developing more effective loss functions specifically for fine-grained vocal features could further improve results. Additionally, while parameter-efficient, the computational complexity of the attention matrix increases with cross-layer concatenation, suggesting a need for further optimization in extremely deep networks. The team plans to test FGNL's effectiveness in computer vision tasks where cross-layer correlation may also prove beneficial.
Summary updated