Introduction
Music retrieval systems have traditionally relied on metadata such as artist names and song titles. However, retrieving music based solely on the audio signal remains a significant challenge, particularly when dealing with variations in performance. This research explores how "tonal" layers—specifically melody, bass line, and harmony—can be leveraged to identify different versions of the same song (version identification) and to power "query-by-humming" (QBH) systems.
The Challenge of Invariant Representation
The core difficulty in content-based music retrieval is finding a representation that is "invariant" to performance changes. Whether a song is performed by a heavy metal band, an acoustic ensemble, or hummed by an amateur, the underlying musical "DNA" must remain detectable. This paper addresses two primary tasks:
- Version Identification: Detecting "cover songs" where instrumentation, tempo, and style may differ drastically.
- Query-by-Humming: Matching a short, often imperfect, monophonic recording of a user singing or humming to a polyphonic studio track.
Methodology: A Unified Tonal Framework
The researchers propose a unified framework using three primary tonal descriptors:
- Harmony: Represented by Harmonic Pitch Class Profiles (HPCP), which measure the intensity of the 12 semitones in the musical scale. This feature captures the overall "color" of a song.
- Melody and Bass Line: Extracted using state-of-the-art pitch tracking. To ensure these features are robust against performance variations, the authors apply a "semitone + octave" abstraction. This process quantizes raw pitch data and collapses it into a single octave (pitch class), creating a "chroma-like" representation for melodic content.
- Sequence Matching ($Q_{max}$): To compare these sequences, the system employs a dynamic programming algorithm ($Q_{max}$) derived from nonlinear time series analysis. This allows for partial matches and handles variations in tempo and song structure.
- Late Fusion: Machine learning classifiers (such as Random Forests and K-Star) are used to weigh the dissimilarity scores from all three descriptors, determining the likelihood that two tracks are versions of the same song.
Experimental Setup and Results
The framework was tested on a substantial collection of 2,125 songs containing 523 version sets. For the QBH task, 118 queries from 17 subjects were matched against the full polyphonic database.
Key Insights
- Harmony as a Baseline: Harmony (HPCP) proved to be the most robust single feature for version identification, achieving a Mean Average Precision (MAP) of 0.698.
- The Power of Fusion: While harmony is reliable, melody and bass lines provide complementary data. Fusing these features pushed classification accuracy to 88.46%. This is particularly useful in cases like a cappella versions where harmonic accompaniment is absent.
- QBH as Version Identification: The authors demonstrate that QBH is essentially a version identification task where the "cover" is a monophonic hummed query. By applying the same abstraction and matching techniques, the system achieved a Mean Reciprocal Rank (MRR) of 0.67 for well-tuned singers, typically placing the correct song in the top two results.
Conclusion
By creating a "common language" through pitch-class abstraction, this research successfully bridges the gap between diverse musical performances and user-generated queries. While the system is highly accurate, future work is needed to address transcription errors in complex mixes and the computational load of the $Q_{max}$ algorithm for commercial-scale databases. Additionally, improving robustness against "de-tuning"—where a user changes key mid-hum—remains a priority for refining query-by-humming technology.
Summary updated