Abstract
Traditional evaluation in Music Information Retrieval (MIR) often relies on a static "ground truth" dataset. However, harmonic analysis is inherently subjective, particularly in pop music where multiple interpretations of the same pitch-class set can be equally valid. This paper presents a Bayesian chord-labelling algorithm and a novel evaluation framework that leverages expert human consensus to measure performance. By acknowledging harmonic ambiguity as a feature rather than a bug, the authors establish a more realistic "ceiling" for algorithmic performance based on human expert disagreement.
Introduction
In the field of Music Information Retrieval (MIR), the standard approach to evaluating algorithms involves comparing output against a gold-standard "ground truth." While this works for objective data, music theory is rarely black and white. A specific set of notes might be interpreted as a C Major chord with an added 6th by one analyst, while another identifies it as an A minor 7th in first inversion.
The authors of this study identify a significant gap in current MIR methodologies: traditional metrics do not account for this inherent subjectivity. If highly-qualified human experts cannot agree on a single "correct" label, penalizing an algorithm for choosing one valid interpretation over another is mathematically and theoretically flawed. This research seeks to move the field toward a model that accounts for harmonic ambiguity and expert disagreement.
The Bayesian Chord-Labelling Algorithm
The researchers introduced a three-module system designed to identify chord roots, types, functional bass notes, and extensions from symbolic MIDI data. The system is built on three primary pillars:
- Bayesian Model Selection: This serves as the core logic of the system. Unlike many models that require pre-segmented audio or MIDI, this algorithm simultaneously handles segmentation (determining the duration of a chord) and labelling (identifying the chord itself).
- Dirichlet Distribution: This statistical tool is used to model the contributions of various pitch classes within a specific temporal window, allowing the model to weigh the importance of different notes.
- Rule-Based Bass Detection: A secondary layer determines the functional bass note by prioritizing notes with longer durations and those situated in lower registers, reflecting how humans perceive the foundation of a chord.
Methodology and Expert Evaluation
To test the algorithm, the authors moved beyond simple songbook comparisons. They recruited four "highly-qualified" academic musicians to perform a multi-dimensional assessment of 40 pop song excerpts. The evaluation framework consisted of three main components:
- MIDI Comparison: Comparing algorithm-generated labels directly against full-score transcriptions.
- Songbook Baseline: Evaluating published songbook labels to see how the algorithm compares to commercially available (and often simplified) interpretations.
- Ear-Only Assessment: Experts performed assessments based on listening alone to simulate real-world musical perception.
By treating each expert's feedback as an individual solution, the researchers could calculate the rate of disagreement among the humans themselves. This created a "human ceiling"; if the algorithm's disagreement with the experts matched the experts' disagreement with each other, the algorithm could be considered to be performing at a human level.
Key Insights: Ambiguity as a Feature
The study highlights that harmonic ambiguity in pop music typically arises from three sources: conflicts in root assignment, the friction between pop styles and classical harmonic theory, and the relative autonomy of the bass line.
A critical finding was that segmentation matters as much as identification. Deciding exactly when a chord changes is a fundamental part of the analytical process that is often overlooked in models that rely on fixed time-windows.
Results and Performance
Preliminary tests of the Bayesian core demonstrated an accuracy of approximately 75% for chord root and type at each beat. While this may seem lower than some "ground truth" benchmarks, the expert coherence study revealed that experts frequently provide different corrections for the same passage. Using a binomial test on the identity of these corrections, the authors statistically demonstrated that for many musical passages, a "single correct solution" simply does not exist. The algorithm’s performance was found to be comparable to the simplified labels found in many commercial songbooks, suggesting its utility in real-world applications.
Limitations and Future Directions
Despite its successes, the model has clear areas for growth:
- Voice-Leading: The current system analyzes sets of pitch classes but does not yet account for how individual notes move between chords—a vital component of human harmonic perception.
- Vocabulary: The algorithm is currently restricted to a specific set of triads and four-note chords (major, minor, sus4, sus9, augmented, and diminished).
- Extensions: The detection of 7ths, 9ths, and 13ths is currently limited to notes of "significant duration," potentially missing brief but coloristically important harmonic elements.
Future work will focus on refining the Bayesian model to better handle the autonomy of the bass note and the complex structures of pop music that often defy traditional classical rules.
Summary updated