Abstract
This paper introduces GraFPrint, an audio identification framework that leverages the structural learning capabilities of Graph Neural Networks (GNNs) to create robust audio fingerprints. Our method constructs a k-nearest neighbor (k-NN) graph from time-frequency representations and applies max-relative graph convolutions to encode local and global information. The network is trained using a self-supervised contrastive approach, which enhances resilience to ambient distortions by optimizing feature representation. GraFPrint demonstrates superior performance on large-scale datasets at various levels of granularity, proving to be both lightweight and scalable, making it suitable for real-world applications with extensive reference databases.
Introduction
Audio identification—the technology enabling services like Shazam—requires matching short, noisy audio snippets against massive reference databases. While traditional "landmark-based" methods have served as the industry standard, they often falter in high-noise environments. Modern deep learning approaches, such as Convolutional Neural Networks (CNNs) and Transformers, have improved robustness but face limitations: CNNs are constrained by rigid grid-like processing, and Transformers are computationally intensive.
GraFPrint addresses these issues by treating spectrograms as graphs rather than simple images or sequences. By modeling the "constellation-like" structure inherent in audio data, GraFPrint captures intricate relationships between acoustic points regardless of their temporal or spectral position, achieving state-of-the-art robustness while remaining significantly more lightweight than Transformer-based alternatives.
Methodology
The GraFPrint framework processes audio through a multi-stage pipeline designed to extract noise-invariant features.
- Feature Extraction: Audio is converted into log-mel spectrograms with added positional encodings to preserve essential time and frequency information.
- Graph Construction: The system treats each time-frequency point as a node. A k-nearest neighbor (k-NN) algorithm connects nodes close in feature space, forming an undirected graph.
- GNN Encoder: The core component utilizes max-relative graph convolutions. This operation enables nodes to aggregate information from neighbors by focusing on significant differences, effectively filtering out background noise.
- Contrastive Training: The model is trained using a self-supervised NT-Xent loss. This approach optimizes the embedding space by pulling together "positive" pairs (the same clip with different augmentations like noise or reverb) and pushing apart "negative" pairs (different songs).
- Search and Retrieval: Efficient identification is achieved using a Product-Quantized Inverted File Index (FAISS) for approximate nearest-neighbor search across the reference database.
Experimental Setup
The researchers evaluated GraFPrint using the Free Music Archive (FMA) dataset. Training was conducted on FMA-small (8,000 clips), while reference databases included FMA-medium (25,000 clips) and FMA-large (106,000 clips). To simulate real-world conditions, the MUSAN dataset provided background noise (e.g., restaurant, street), and the Aachen Room Impulse Response Database was used to simulate room reverberation.
Key Results and Discussion
GraFPrint consistently outperformed CNN and Transformer-based baselines, particularly in challenging acoustic environments.
- Robustness to Noise: For 1-second queries at 20dB SNR, GraFPrint achieved a 98.8% top-1 hit rate, surpassing NAFP (76.9%) and Transformer (90.9%) baselines.
- Resilience to Reverberation: While most models degrade significantly under reverb, GraFPrint’s performance dropped by only 11.6 percentage points, compared to a 21.8-point drop for the best baseline. This suggests the GNN successfully captures the underlying "shape" of the sound.
- Scalability and Efficiency: On the FMA-large database, GraFPrint outperformed the Audio Spectrogram Transformer (AST) baseline by at least 20.5 percentage points. Crucially, GraFPrint uses only 18 million parameters, making it much more efficient than the 45-million-parameter AST.
Conclusion
GraFPrint demonstrates that graph-based modeling is highly effective for audio identification, providing superior robustness and efficiency. While the dynamic construction of k-NN graphs currently impacts training speed, future work will focus on optimizing graph construction and integrating learned structures into data-driven hashing methods to further accelerate retrieval in billion-scale databases.
Summary updated