Enhancing Neural Audio Fingerprint Robustness to Audio Degradation for Music Identification

Recep Oguz Araz, Guillem Cortès-Sebastià, Emilio Molina, Joan Serrà, Xavier Serra, Yuki Mitsufuji, Dmitry Bogdanov

Abstract

Audio fingerprinting (AFP) allows the identification of unknown audio content by extracting compact representations that remain robust against common audio degradations. Neural AFP methods often employ metric learning, where representation quality is influenced by the nature of the supervision and the utilized loss function. However, recent work unrealistically simulates real-life audio degradation during training, resulting in sub-optimal supervision. In this work, we propose a series of best practices to enhance self-supervision by leveraging musical signal properties and realistic room acoustics. We present a systematic evaluation of various metric learning approaches, demonstrating that a self-supervised adaptation of the triplet loss yields superior performance. Our approach, NMFP, achieves state-of-the-art performance on both synthetically degraded and real-world datasets.

Introduction

Audio Fingerprinting (AFP) is the core technology behind music identification services. While modern neural AFP methods are powerful, they often struggle with "real-world" degradation, such as noise and reverberation in crowded venues like bars or concert halls. Current research often relies on unrealistic training simulations and defaults to the NT-Xent loss without exploring alternatives. This paper addresses these issues by introducing NMFP (Neural Music Fingerprinting), which incorporates realistic acoustic modeling and optimized metric learning strategies.

Methodology and Best Practices

The authors identify and address three major gaps in current AFP research: unrealistic training, sub-optimal loss functions, and evaluation flaws. To harden the model against degradation, several "best practices" were introduced:

  1. Realistic Reverberation: Using full Room Impulse Responses (IRs) and simulating "past reverberation" to account for the temporal context of audio signals.
  2. Eliminating False Negatives (FN): Ensuring each training batch contains only one segment per track to provide a cleaner learning signal.
  3. Frequency Range Expansion ($F_{LC}$): Lowering the frequency cut-off from 300 Hz to 160 Hz to leverage resilient bass cues.
  4. Metric Learning Exploration: A systematic comparison of various self-supervised losses, including NT-Xent, DCL, KCL, A&U, and Triplet loss.

The impact of these enhancements is detailed in the ablation study below, showing significant gains in real-world performance as each feature is added.

Ablation study of proposed training enhancements showing incremental improvements in Top-1 hit rate on synthetic and real-world data.

Key Insights: The Triplet Loss and Acoustic History

A significant finding is the superiority of the triplet loss (with hard positive and semi-hard negative mining) over the currently popular NT-Xent loss for the AFP task. The triplet loss provided a 4.7% performance boost on real-world data. Furthermore, accounting for the "acoustic history" (reverb tails from previous audio segments) significantly improved robustness in cavernous environments.

The study also revealed that batch construction is critical: while adding multiple positives per anchor hindered NT-Xent performance, it benefited the triplet loss, highlighting that loss functions require tailored batching strategies.

Results

NMFP sets a new state-of-the-art for open-source neural audio fingerprinting. On 1-second clips recorded in noisy real-world venues, NMFP achieved a 63.9% Top-1 hit rate, nearly doubling the performance of the baseline. For longer 10-second queries, accuracy reaches 84.6%, outperforming previous state-of-the-art models like NAFP-LAMB by over 8%.

Comparison of NMFP against NAFP baselines on real-world recordings, demonstrating superior performance across various query lengths.

Conclusion

By refining training simulations and optimizing metric learning, NMFP significantly enhances the robustness of neural audio fingerprints. Future work will explore the transition to binary representations for increased memory efficiency and the integration of more advanced neural architectures like Transformers.

Summary updated

Abstract

Audio fingerprinting (AFP) allows the identification of unknown audio content by extracting compact representations, termed audio fingerprints, that are designed to remain robust against common audio degradations. Neural AFP methods often employ metric learning, where representation quality is influenced by the nature of the supervision and the utilized loss function. However, recent work unrealistically simulates real-life audio degradation during training, resulting in sub-optimal supervision. Additionally, although several modern metric learning approaches have been proposed, current neural AFP methods continue to rely on the NT-Xent loss without exploring the recent advances or classical alternatives. In this work, we propose a series of best practices to enhance the self-supervision by leveraging musical signal properties and realistic room acoustics. We then present the first systematic evaluation of various metric learning approaches in the context of AFP, demonstrating that a self-supervised adaptation of the triplet loss yields superior performance. Our results also reveal that training with multiple positive samples per anchor has critically different effects across loss functions. Our approach is built upon these insights and achieves state-of-the-art performance on both a large, synthetically degraded dataset and a real-world dataset recorded using microphones in diverse music venues.
BibTeX
@inproceedings{araz2025enhancing,
  author = {Recep Oguz Araz and Guillem Cortès-Sebastià and Emilio Molina and Joan Serrà and Xavier Serra and Yuki Mitsufuji and Dmitry Bogdanov},
  title = {{Enhancing Neural Audio Fingerprint Robustness to Audio Degradation for Music Identification}},
  booktitle = {ISMIR},
  year = {2025},
  url = {https://openreview.net/forum?id=mvGg5CeJSs},
}
Figure