Polyphonic Music Note Onset Detection Using Semi-Supervised Learning

Wei You, Roger B. DannenbergISMIR 2007

Introduction

Note onset detection—the precise identification of when a musical note begins—is a foundational task for music information retrieval (MIR) applications like automatic transcription and tempo estimation. While relatively straightforward for solo instruments, the task becomes "massively polyphonic" and notoriously difficult in orchestral settings where dozens of instruments overlap.

The primary barrier to progress in this field is the "data bottleneck." Hand-labeling a single movement of a symphony can require over 30 hours of meticulous work. Without large-scale labeled datasets, modern machine learning models cannot be trained effectively. This research addresses the problem by leveraging semi-supervised learning to generate training labels automatically from existing MIDI files.

The Bootstrapping Pipeline

The core of this approach is a "bootstrap" pipeline designed to transform noisy, automated labels into high-quality training data. The process begins by aligning standard MIDI scores to real audio recordings using chroma features and dynamic programming. This provides a rough initial estimate of onset times.

The proposed semi-supervised bootstrapping pipeline for onset detection, illustrating the iterative relabeling and training process.

As shown in the system architecture above, the pipeline follows an iterative refinement process:

  1. Initial Training: A Support Vector Machine (SVM) is trained on the rough MIDI-aligned labels.
  2. Prediction and Relabeling: The model predicts onsets on the training set. The system then "relabels" the data by identifying the highest probability peak within a small window (250ms) of the original MIDI estimate.
  3. Iteration: The SVM is retrained on these refined labels. This cycle typically repeats 8–10 times until the model converges, effectively allowing the system to "search" for the true acoustic onset in the neighborhood of the noisy MIDI estimate.

Experimental Setup

The model was trained on 10 polyphonic classical pieces from the RWC Music Database, encompassing approximately 90 minutes of audio and 30,000 distinct onset times from composers such as Bach, Beethoven, and Tchaikovsky. Evaluation was performed on a separate performance of Strauss’s Blue Danube.

Feature extraction utilized standard acoustic features (Spectral Flux and Phase Deviation) from the Aubio library, alongside experimental "Chroma Flux" features intended to capture melodic transitions.

Performance Analysis

The results demonstrate that the semi-supervised approach significantly outperforms traditional hand-tuned algorithms. The trained SVM achieved a best F-measure of 0.47, a substantial improvement over the Aubio baseline's score of 0.38.

Precision-Recall curves comparing the SVM-based approach against the Aubio baseline and various feature configurations.

As illustrated in the Precision-Recall curves, the bootstrapping process is critical to success. A model trained on a small set of 100 hand-labeled onsets without bootstrapping (labeled "NoBootstrap") performed abysmally, with an F-measure of only 0.03. This confirms that the sheer volume of automatically generated data, even if initially noisy, is what makes the system viable. Interestingly, the research found that the classifier's iterative refinement was more impactful than adding complex new features like higher-order differences or larger window sizes.

Conclusion and Future Work

This study proves that MIDI files can serve as a powerful resource for generating massive training sets without manual human intervention. However, the system remains sensitive to the initial audio-to-score alignment; if the alignment error exceeds 100ms, the labels become too noisy for effective learning. Future work will focus on improving alignment robustness and exploring whether specialized detectors for specific orchestral textures (e.g., strings vs. brass) can further enhance accuracy.

Summary updated

BibTeX
@inproceedings{you2007polyphonic,
  author = {Wei You and Roger B. Dannenberg},
  title = {{Polyphonic Music Note Onset Detection Using Semi-Supervised Learning}},
  booktitle = {ISMIR},
  year = {2007},
}
Figure