Introduction
In the field of Music Information Retrieval (MIR), melody extraction involves identifying the dominant pitch sequence in complex polyphonic audio. Traditional systems often struggle with "octave errors"—misidentifying a frequency as double or half the true note—and interference from low-frequency accompaniment. Most algorithms search a broad frequency range (typically 80Hz to 1280Hz), which often introduces false positives. This paper addresses these issues by introducing a harmonic structure model refined by harmonic fitness and dynamic range estimation.
Methodology
The proposed system operates in three distinct stages to ensure robust pitch detection.
1. Candidate Extraction
The algorithm first calculates the weight of various fundamental frequency ($F0$) candidates using a harmonic structure model. This model uses Gaussian distributions to represent the expected energy at the fundamental frequency and its integer multiples (harmonics) in the frequency domain. By performing an inner-product between this model and the audio's Short-Time Fourier Transform (STFT) spectrum, the system identifies potential pitches.
2. Harmonic Fitness Refinement
To mitigate octave errors, the authors introduce a "harmonic fitness" score. Instead of relying solely on total energy, the algorithm verifies the presence of expected harmonic partials. If a candidate pitch lacks its corresponding "family" of harmonics in the actual audio spectrum, its weight is penalized. This ensures that mathematical artifacts or interference are not mistaken for the melody.
3. Melody Range Estimation
A key innovation of this paper is the dynamic estimation of the melody's frequency range. The algorithm identifies "reliable" candidates—those that remain stable (within $\pm100$ cents) for at least 20 frames. By analyzing the distribution of these stable notes, the system calculates a tighter, clip-specific frequency range. This pruning of the search space prevents the algorithm from selecting outliers in extreme frequency registers.
Evaluation and Results
The algorithm was benchmarked against the ADC 2004 and MIREX 2005 datasets using Raw Pitch Accuracy (RPA) and Raw Chroma Accuracy (RCA).
- Performance: The method achieved an RPA of 80.8% and an RCA of 86.7% on the ADC 2004 dataset.
- Comparative Success: It significantly outperformed established baselines, such as models by Ellis (73.2% RPA) and Ryynänen (68.6% RPA).
- Optimization: A frame length of 64ms was found to be optimal for balancing temporal resolution and frequency precision.
Conclusion
By combining a structured harmonic model with dynamic range constraints, the proposed system effectively reduces common errors in polyphonic melody extraction. While the reliance on candidate stability may pose challenges for highly virtuosic or fragmented melodies, the approach offers a significant step forward in robust pitch estimation. Future work will focus on further refining the harmonic fitness calculations to improve performance in high-noise environments.
Summary updated