Abstract
This paper presents a bootstrap learning framework for music onset detection, addressing the scarcity of manually labeled acoustic data. By leveraging global audio-to-score alignment via Dynamic Time Warping (DTW) to generate initial noisy labels and iteratively refining them through a neural network, the system bridges the gap between synthetic and real-world performance data. Experimental results on monophonic and polyphonic recordings demonstrate significant improvements in both miss rates and temporal precision.
Introduction
In Music Information Retrieval (MIR), "onset detection"—identifying the exact start of a musical note—is a fundamental building block for tasks like beat tracking and automated transcription. While supervised machine learning models, such as Multi-Layer Perceptrons (MLP), are highly effective, they are notoriously data-hungry.
The primary bottleneck is the acquisition of "ground truth" labels. Manually marking every note onset in a recording is an exhausting, multi-hour chore for human experts. Consequently, researchers often train models on synthetic audio where onset times are known. However, these models frequently struggle with the nuances of real-world acoustic performances, such as slurred notes, room resonance, and varying instrument timbres.
Methodology
The authors propose a "bootstrap learning" method that eliminates the need for manual labeling by bridging the gap between symbolic music (MIDI) and acoustic recordings through an iterative loop.
1. Audio-to-Score Alignment
The system begins with a MIDI file of the piece. It converts both the MIDI and the target acoustic audio into chromagrams—12-element vectors representing the intensity of the 12 semitones. It then employs Dynamic Time Warping (DTW) to find a global "path" that matches the MIDI to the audio, providing a rough estimate of where each note starts.
2. The Bootstrap Loop
The core of the system is an MLP neural network that analyzes local acoustic features, including logarithmic energy, fundamental frequency (F0), harmonic strengths, and zero-crossing rates. The learning process follows these steps:
- Initialization: The MLP is first trained on synthetic data to establish a baseline intelligence.
- Refinement: The system runs the detector on real audio. It combines the detector’s output with a probability function derived from the DTW alignment. Essentially, the DTW provides a global "map" while the neural network provides local precision.
- Relabeling & Retraining: The point where these two signals most strongly agree becomes the new "ground truth" label. The MLP is then retrained on these refined labels. This process repeats (typically 6–10 times) until the model converges.
Key Results
The improvements were dramatic, particularly for real acoustic recordings where synthetic-only models typically fail.
- Monophonic (Trumpet): The baseline model (no bootstrapping) exhibited a 15% miss rate and an average timing error of 35ms. After bootstrapping, the miss rate plummeted to 2% and the error dropped to just 8ms.
- Polyphonic (Piano): In complex multi-note environments, the bootstrapped model reduced the miss rate from 16% to 5% and improved timing accuracy from 27ms to 11ms.
- Precision: Because the model is guided by a score, it is significantly less likely to "hallucinate" notes that do not exist compared to blind detectors.
Conclusion
The "aha moment" of this research is the synergy between global and local strengths. DTW alignment is excellent at global correspondence but has poor temporal resolution. Conversely, neural networks excel at local precision but can get lost in acoustic noise without a map. By using the score to guide the neural network, the bootstrap process successfully teaches the model the characteristics of real acoustic instruments without requiring human intervention. While the system currently requires a machine-readable score for the training phase, it represents a major step toward high-precision, automated music transcription.
Summary updated