GraFPrint: A GNN-Based Approach for Audio Identification

Aditya Bhattacharjee, Shubhr Singh, Emmanouil Benetos

TL;DR

GraFPrint is a novel audio identification framework that treats spectrograms as graphs rather than simple images or sequences. By using Graph Neural Networks (GNNs) to model the relationships between time-frequency points, the system achieves state-of-the-art robustness against noise and reverberation while remaining significantly more lightweight than Transformer-based alternatives.

What Problem Does This Paper Address?

Audio identification—the technology behind apps like Shazam—requires matching a short, often noisy snippet of music against a massive database of reference recordings. While "landmark-based" methods (which look for peaks in spectrograms) have been the industry standard for decades, they can struggle in extremely noisy environments.

Recent deep learning approaches using Convolutional Neural Networks (CNNs) and Transformers have improved performance by learning noise-invariant features. However, CNNs are restricted by their rigid grid-like processing, and Transformers can be computationally expensive. The authors of GraFPrint argue that audio data possesses an inherent "constellation-like" structure that is better captured by graphs, allowing the model to learn the intricate relationships between key acoustic points regardless of where they appear in time or frequency.

Methods

The GraFPrint framework consists of several key stages:

  1. Feature Extraction: The system converts audio into log-mel spectrograms and adds positional encodings to preserve time and frequency information.
  2. Graph Construction: Instead of a standard grid, the model treats each time-frequency point as a node. It uses a k-nearest neighbor (k-NN) algorithm to connect nodes that are close to each other in the feature space, creating an undirected graph.
  3. GNN Encoder: The heart of the system is a GNN that uses max-relative graph convolutions. This operation allows nodes to aggregate information from their neighbors by focusing on the most significant differences, which helps the model ignore "background" noise.
  4. Contrastive Training: The model is trained using a self-supervised contrastive approach (NT-Xent loss). It learns to pull "positive" pairs (the same audio clip with different augmentations like noise or reverb) closer together in the embedding space while pushing "negative" pairs (different songs) further apart.
  5. Search and Retrieval: For the final identification, the system uses FAISS (Product-Quantized Inverted File Index) to perform an efficient approximate nearest-neighbor search across the reference database.

Datasets

The researchers evaluated GraFPrint using the Free Music Archive (FMA) dataset:

  • Training: FMA-small (8,000 30-second clips).
  • Reference Databases: FMA-medium (25,000 clips) and a scaled-up FMA-large (106,000 clips).
  • Augmentations: To simulate real-world conditions, they used the MUSAN dataset for background noise (restaurant, street, etc.) and the Aachen Room Impulse Response Database to simulate room reverberation.

Key Insights and Ideas

  • Spectrograms as Constellations: Inspired by classic landmarking, the authors treat the spectrogram as a "constellation map." By using a graph, the model can capture non-Euclidean relationships that CNNs might miss.
  • Resilience to Reverb: One of the most impressive findings is GraFPrint's stability. While most models see a massive performance drop when reverberation is added, GraFPrint’s performance is remarkably stable, suggesting the GNN is capturing the underlying "shape" of the sound rather than just the surface-level textures.
  • Efficiency Matters: GraFPrint uses only 18 million parameters, compared to the 45 million used by the state-of-the-art Audio Spectrogram Transformer (AST) baseline. This makes it much more suitable for deployment in large-scale, real-world applications.

Key Results

GraFPrint consistently outperformed both CNN and Transformer-based baselines, particularly in difficult conditions:

  • Noisy Environments: For 1-second queries at 20dB SNR, GraFPrint achieved a 98.8% top-1 hit rate, significantly higher than the NAFP (76.9%) and Transformer (90.9%) baselines.
  • The Reverb Test: When convolutional reverb was added to 1-second queries, GraFPrint's performance dropped by only 11.6 percentage points, whereas the best baseline dropped by 21.8 percentage points.
  • Large-Scale Scaling: On the FMA-large database (106k segments), GraFPrint outperformed the AST baseline by a minimum of 20.5 percentage points across all tested signal-to-noise ratios.

Limitations and Future Work

The primary drawback noted by the authors is training speed. Dynamically constructing and updating the k-NN graph for every batch is computationally expensive, which slows down the training process as the number of nodes or the dataset size increases.

Future work will likely focus on:

  • Developing more efficient graph construction methods to speed up training.
  • Exploring how the learned graph structure can be integrated directly into data-driven hashing methods to further accelerate the search and retrieval process in billion-scale databases.

Summary updated

Abstract

This paper introduces GraFPrint, an audio identification framework that leverages the structural learning capabilities of Graph Neural Networks (GNNs) to create robust audio fingerprints. Our method constructs a k-nearest neighbour (k-NN) graph from time-frequency representations and applies max-relative graph convolutions to encode local and global information. The network is trained using a self-supervised contrastive approach, which enhances resilience to ambient distortions by optimizing feature representation. GraFPrint demonstrates superior performance on large-scale datasets at various levels of granularity, proving to be both lightweight and scalable, making it suitable for real-world applications with extensive reference databases.
BibTeX
@inproceedings{bhattacharjee2025grafprint,
  author = {Aditya Bhattacharjee and Shubhr Singh and Emmanouil Benetos},
  title = {{GraFPrint: A GNN-Based Approach for Audio Identification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=5znF98xnJo},
}

Loading related content...

Figure