TL;DR
This paper introduces a lightweight, two-stage framework for Automatic Sample Identification (ASID) that matches the performance of state-of-the-art models while using only 9% of the parameters. By combining a self-supervised Graph Neural Network (GNN) encoder with a novel cross-attention classifier, the system effectively identifies musical samples even when they have been heavily pitch-shifted, time-stretched, or buried under new instrumentation.
What Problem Does This Paper Address?
Identifying "samples"—portions of existing recordings reused in new songs—is significantly harder than standard audio fingerprinting (like Shazam). While standard fingerprinting excels at finding exact matches in noisy environments, ASID must contend with sampling transformations:
- Creative Manipulation: Samples are often pitch-shifted, time-stretched, and processed with heavy effects.
- Musical Interference: The sample is usually layered with new drums, vocals, or instruments, masking the original source.
Existing deep learning approaches often rely on massive, computationally expensive Convolutional Neural Networks (CNNs). The authors aim to create a more efficient, scalable system that remains robust to these artistic modifications.
Methods
The authors propose a two-stage retrieval pipeline:
- The Encoder (GNN): Instead of a standard CNN, they adapt a Graph Neural Network architecture. It treats log-Mel spectrograms as a set of time-frequency points (nodes) and learns structural relationships between them. This model is trained using self-supervised contrastive learning (NT-Xent loss), where the model learns to map a "clean" sample and its "transformed/mixed" version to similar points in an embedding space.
- The Refiner (Cross-Attention Classifier): To improve retrieval accuracy, the authors introduce a multi-head cross-attention (MHCA) classifier. After an initial fast search (Approximate Nearest Neighbor), this classifier takes the query and the top candidates and performs a fine-grained comparison to reject "false alarms" and re-rank the results.
- Data Augmentation: To simulate real-world production, they used source separation (Demucs) to isolate stems (drums, bass, etc.) from the Free Music Archive. They then synthetically created query-reference pairs by applying random pitch-shifting (±3 semitones), time-stretching (70-150%), and gain variations.
Datasets
- Training: The Free Music Archive (FMA) medium dataset (25,000 tracks).
- Evaluation: The Sample100 dataset, which contains real-world hip-hop queries and their original sources.
- New Contribution: The authors released an extended version of Sample100 with expert-level, fine-grained temporal annotations (precise start/end times), time-stretching estimates, and instrumentation labels. This allows for much more granular "segment-wise" evaluation than previous versions of the dataset.
Key Insights and Ideas
- Efficiency via Graphs: GNNs are highly effective at capturing local and global interactions in audio data while being much smaller than deep CNNs. The proposed model has 20M parameters, compared to the 222M in the previous state-of-the-art ResNet50-IBN.
- The Power of Negatives: The authors found that increasing the batch size during contrastive learning (from 256 to 1024) improved the Mean Average Precision (mAP) by nearly 7 percentage points. Larger batches provide more "hard negatives," forcing the model to learn finer distinctions between similar-sounding but different tracks.
- Two Stages are Better Than One: The cross-attention classifier added a 2.6pp boost to mAP. It acts as a "sanity check" that can look at the latent relationships between the query and candidate in a way a simple distance metric cannot.
Key Results
- Performance: The model achieved a Mean Average Precision (mAP) of 44.2%, slightly edgeing out the much larger ResNet50-IBN (44.1%).
- Short Query Robustness: On 20-second queries, the system showed a 14.1 percentage point increase in Top-1 hit rate over the reimplemented baseline.
- Sample Type Sensitivity: The system performed better on melodic/harmonic "riffs" (0.471 mAP) than on percussive "beats" (0.391 mAP). The authors suggest this is because drum samples are often overdubbed or buried deeper in the mix, making them harder for the GNN to isolate.
Limitations and Future Work
- The Time-Stretch Wall: There is a significant performance gap based on the degree of manipulation. Samples with minimal time-stretching (<5%) had an mAP of 0.503, while those with substantial stretching (>5%) dropped to 0.340. Large tempo changes fundamentally alter the temporal relationships the GNN relies on.
- Masking: Heavily masked samples (where the new music is much louder than the sample) remain a challenge.
- Future Directions: The authors suggest exploring end-to-end training (rather than two-stage) and incorporating source separation directly into the inference pipeline to "unmask" samples before identification.
Summary updated