Abstract
Extracting the singing melody from polyphonic audio remains a significant challenge due to interference from background instruments. This paper presents a novel harmonic tracking strategy that prioritizes temporal continuity over isolated frame-by-frame analysis. By identifying stable "seed" segments and tracking them using subharmonic summation and timbre-based identity verification, the proposed method achieves an overall accuracy of 82.23% on the LabROSA database, demonstrating a clear advantage over traditional predominant pitch estimation techniques.
Introduction
In Music Information Retrieval (MIR), melody extraction involves isolating the fundamental frequency ($F_0$) of the lead instrument or voice from a complex mixture of sounds. The primary obstacle is "pollution"—instances where non-melody instruments (e.g., drums, guitars) temporarily dominate the spectral energy. Traditional algorithms often fail in these scenarios because they treat each time frame independently. When a frame is obscured by noise or a loud transient, the algorithm loses the melody's trajectory. This research addresses the problem by introducing a tracking mechanism that maintains the "thread" of the melody through time.
Methodology
The proposed system utilizes a multi-stage pipeline designed to transition from local spectral analysis to global temporal tracking.
1. Spectral Representation and Initial Estimation
The process begins with Subharmonic Summation (SHS), which sums the power of a frequency and its harmonics to generate a pitch likelihood map. This provides a robust foundation for identifying the fundamental frequency even in noisy environments. From this map, initial predominant $F_0$ candidates are estimated.
2. Seed Detection and Harmonic Tracking
Rather than accepting every $F_0$ estimate, the algorithm searches for "stable harmonic structures"—sequences of pitches that remain consistent for a minimum duration. These stable segments serve as anchors or "seeds." From these seeds, the algorithm tracks the melody both forward and backward in time.
3. Identity Verification
To prevent the tracker from jumping to a different instrument playing a similar pitch, an "Identity Verification" step is employed. The system calculates the correlation of relative amplitudes (representing timbre) between the current frame and the established melody track. If the spectral "fingerprint" matches, the track continues; otherwise, it is terminated.
4. Conflict Resolution
In cases where multiple tracked structures overlap, a saliency-based competition is used. The more dominant (higher saliency) track is retained, while the weaker "intruder" is discarded, resulting in a single, continuous melody stream.
Results
The tracking method was evaluated against the LabROSA database and a specialized vocal-only test set (including ISMIR 2004 tracks). The results indicate that the proposed harmonic tracking method consistently outperforms the baseline predominant pitch estimation.
As shown in the table above, on the full LabROSA database, the system achieved an accuracy of 82.23%, compared to 78.30% for the baseline. For the vocal-only subset, the improvement was even more pronounced, reaching 79.39% (a 5.27% increase over the baseline). Beyond the numerical gains, the tracking approach produces more musically coherent pitch sequences by organizing them into logical harmonic units.
Discussion and Conclusion
The core insight of this work is that temporal continuity is essential for robust melody extraction. By leveraging the history of the melody and using timbre as a filter, the system can bridge gaps where the signal is temporarily obscured.
While effective, the current verification and conflict resolution rules are relatively simple. Future work will focus on developing more sophisticated timbre models to handle instruments with similar harmonic profiles and refining the mapping algorithms for complex overlapping tracks. Additionally, making the stability threshold adaptive to the musical context could further enhance performance.
Summary updated