Abstract
Methods based on moments and moment invariants have been extensively used in image analysis tasks but rarely in audio applications. However, while images are typically two-dimensional (2D) and audio signals are one-dimensional (1D), many studies have showed that image analysis techniques can be successfully applied on audio after 1D audio signal is converted into a 2D time-frequency auditory image. Motivated by these observations, in this chapter we propose using moments to solve an important problem of audio analysis, i.e., music identification.
Introduction
In the modern digital landscape, music identification services like Shazam have become ubiquitous. Most of these services operate on raw, uncompressed audio (Pulse Code Modulation or PCM). However, the vast majority of music is stored and transmitted in compressed formats, primarily MP3. Traditional identification methods require fully decompressing these files before analysis, a process that is computationally expensive and inefficient for large-scale server operations.
Furthermore, "compressed-domain" identification faces significant challenges regarding robustness. Real-world distortions—such as recording a song in a noisy cafe, playing it through low-quality speakers with heavy equalization, or slight changes in playback speed—can drastically alter a song's digital fingerprint. This research proposes a method that treats audio as a visual pattern, utilizing Zernike moments to create a fingerprint that is both efficient to extract and remarkably resilient to distortion.
Methodology: From Bitstream to Auditory Image
The proposed system bridges the gap between audio processing and computer vision by treating the time-frequency representation of a song as a grayscale image. The pipeline consists of several key stages:
1. Feature Extraction and Image Construction
Instead of fully decoding the MP3 bitstream, the algorithm extracts the Modified Discrete Cosine Transform (MDCT) coefficients directly. To form a 2D "auditory image," the researchers group 90 consecutive "granules" (representing approximately 2.3 seconds of audio) across 192 frequency sub-bands. This creates a matrix where one axis represents time and the other represents frequency.
2. Zernike Moments
Once the auditory image is constructed, the system applies Zernike moments—orthogonal polynomials defined on a unit disk. In computer vision, Zernike moments are prized for their rotation invariance and noise robustness. The researchers specifically focus on low-order moments (order $\le$ 2). These low-order moments capture the "global shape" or structural silhouette of the sound's energy distribution, effectively filtering out high-frequency noise and compression artifacts that typically plague identification systems.
3. Binary Fingerprinting
To facilitate rapid database searching, these moments are converted into a binary fingerprint. By comparing the magnitude of moments between consecutive blocks, the system records a '1' if the magnitude increases and a '0' if it decreases. This results in a compact bitstream that can be compared using simple Hamming distance (Bit Error Rate).
Computational Efficiency
By operating directly in the MDCT domain, the algorithm bypasses the most processor-intensive stages of the standard MP3 decoding process. Specifically, it avoids the Inverse MDCT (IMDCT) and the Synthesis Filter Bank, which are required to turn compressed data back into audible PCM waves. This makes the approach highly suitable for high-throughput server environments.
Experimental Evaluation
The system was rigorously tested against a massive library of 21,185 MP3 songs. A query set of nearly 300,000 segments was subjected to 13 different types of distortions to simulate real-world conditions.
- General Accuracy: The system achieved a top-5 hit rate exceeding 90% across the majority of distortion types.
- High Robustness: For common distortions like echo, volume fluctuations, and equalization, the hit rate approached 100%.
- Noise Resilience: Even at a Signal-to-Noise Ratio (SNR) of 15dB, the system maintained high identification accuracy.
- False Positives: In tests against "out-of-database" queries, the false positive rate remained exceptionally low at $3.6 \times 10^{-3}$.
Conclusion and Future Work
The use of low-order Zernike moments proves that a song's identity is effectively encoded in its broad geometric energy distribution. While the system is highly robust, it does face limitations with Time Scale Modification (TSM)—changes in playback speed that do not affect pitch. Because TSM disrupts the fixed timing of the MDCT granules, performance drops when speed changes exceed $\pm 3\%$.
Future research may explore "information fusion," combining Zernike moments with chroma-based features (which track musical notes) to create a hybrid system that is even more resilient to temporal shifts and complex distortions.
Summary updated