Abstract
Automatic sample identification (ASID)—the detection and identification of audio recordings reused in new musical works—is a critical yet challenging task in audio retrieval. Unlike standard audio fingerprinting, which excels at finding exact matches in noisy environments, ASID must remain robust to significant musical modifications such as time-stretching, pitch-shifting, and heavy layering with new instrumentation. In this work, we propose a lightweight and scalable two-stage framework. Our architecture employs a Graph Neural Network (GNN) within a contrastive learning framework, achieving a mean average precision (mAP) of 44.2% while using only 9% of the trainable parameters compared to current state-of-the-art systems. To enhance retrieval quality, we introduce a cross-attention classifier that rejects irrelevant matches and refines candidate rankings. Additionally, we provide fine-grained annotations for the Sample100 dataset to benchmark performance on short, real-world queries.
Introduction
The practice of sampling is a cornerstone of modern music production, yet identifying these samples computationally remains difficult. Traditional audio fingerprinting systems, like Shazam, are designed to identify identical recordings under "real-world" noise conditions. However, they often fail when a sample has been creatively manipulated. In ASID, the query is frequently pitch-shifted, time-stretched, or buried under new drums and vocals, creating a "musical interference" problem.
Existing deep learning solutions typically rely on massive Convolutional Neural Networks (CNNs). While effective, these models are computationally expensive and difficult to scale. This paper introduces a more efficient approach that leverages the structural modeling capabilities of Graph Neural Networks to identify samples with high precision and low overhead.
Methodology
1. The GNN Encoder
Instead of treating audio as a fixed grid (as in CNNs), our model represents log-Mel spectrograms as a set of time-frequency points, or nodes, in a graph. The GNN learns the structural relationships between these points, allowing it to capture both local and global audio features.
We train this encoder using self-supervised contrastive learning (NT-Xent loss). During training, the model is presented with a "clean" sample and a "transformed" version (augmented with pitch shifts, time stretches, and background music). The model learns to map these related pairs to similar points in an embedding space while pushing unrelated samples further apart.
2. Two-Stage Retrieval and Refinement
To balance speed and accuracy, we implement a two-stage pipeline:
- Stage 1 (Candidate Selection): A fast Approximate Nearest Neighbor (ANN) search identifies potential matches from a large database.
- Stage 2 (Refinement): A Multi-Head Cross-Attention (MHCA) classifier performs a fine-grained comparison between the query and the top candidates. This stage acts as a "sanity check," rejecting false positives and re-ranking the results based on latent relationships that a simple distance metric might miss.
3. Data Augmentation and Training
To simulate the complexity of real-world music production, we utilized the Free Music Archive (FMA) medium dataset. We applied source separation (using Demucs) to isolate stems and then synthetically created query-reference pairs. These pairs were subjected to random pitch-shifting (±3 semitones), time-stretching (70-150%), and varying gain levels to ensure the model could handle extreme artistic modifications.
Evaluation and Dataset Contributions
We evaluated our system using the Sample100 dataset, which consists of real-world hip-hop queries and their original sources. As part of this work, we are releasing an extended version of Sample100 featuring expert-level temporal annotations. These include precise start/end times, time-stretching estimates, and instrumentation labels, enabling a more granular "segment-wise" evaluation than was previously possible.
Key Results
Efficiency and Accuracy
Our model achieved a Mean Average Precision (mAP) of 44.2%, slightly surpassing the previous state-of-the-art ResNet50-IBN (44.1%). Crucially, our model achieves this with only 20 million parameters, compared to the 222 million parameters required by the ResNet baseline. This 91% reduction in size makes the system significantly more scalable for large-scale industrial applications.
Robustness to Short Queries
Real-world sample queries are often brief. In tests with 20-second queries, our system demonstrated a 14.1 percentage point increase in Top-1 hit rate over the baseline. The inclusion of the cross-attention refiner alone provided a 2.6 percentage point boost to the overall mAP.
Performance by Sample Type
The system proved more adept at identifying melodic or harmonic "riffs" (0.471 mAP) than percussive "beats" (0.391 mAP). This discrepancy likely stems from the fact that drum samples are frequently overdubbed or heavily masked by new rhythmic elements, making them harder for the GNN to isolate from the background mix.
Limitations and Future Work
While the GNN approach is highly efficient, it faces challenges with extreme temporal manipulation. Performance dropped from an mAP of 0.503 for samples with minimal time-stretching (<5%) to 0.340 for those with substantial stretching (>5%). Large tempo changes alter the temporal relationships the GNN relies on for identification.
Future research will focus on making the architecture more resilient to these temporal shifts and exploring the integration of source separation directly into the inference pipeline to "unmask" samples before they are processed by the encoder.
Summary updated