A Bootstrap Method for Training an Accurate Audio Segmenter

Ning Hu, Roger B. DannenbergISMIR 2005

Introduction

In Music Information Retrieval (MIR), accurately identifying note onsets and offsets—segmentation—is a fundamental task. While supervised models like neural networks excel at this, they typically require vast amounts of manually labeled data. Labeling a single five-minute recording can take hours, creating a significant bottleneck for researchers. This paper presents a "bootstrap" training method that bypasses manual labeling by leveraging coarse audio-to-score alignment to iteratively refine a high-precision segmenter.

Methodology: Bridging Global Alignment and Local Precision

The proposed system integrates two distinct MIR tasks: global alignment and local segmentation.

Audio-to-Score Alignment

The process begins with Dynamic Time Warping (DTW) to match a symbolic MIDI score to an acoustic recording. Both signals are converted into Chroma representations—12-element vectors representing pitch class energy. While DTW effectively identifies the general temporal location of notes, its resolution is constrained by the analysis window (typically 50ms), which is insufficient for precise onset detection.

The Segmentation Model

For local precision, the authors employ a multi-layer feed-forward neural network. Unlike the coarse alignment phase, this model utilizes high-resolution acoustic features, including log energy, fundamental frequency, harmonic strengths, and zero-crossing rates. It outputs a likelihood value (0 to 1) for note onsets.

Acoustic features of a trumpet note showing attack, sustain, and decay phases.

The Bootstrap Loop

The core innovation is an iterative loop that uses the coarse alignment to "teach" the high-resolution model:

  1. PDF Generation: A Gaussian Probability Density Function (PDF) is created based on the initial DTW alignment, serving as a temporal "hint" for likely onset locations.
  2. Constrained Search: The neural network's output is multiplied by this PDF. The system then identifies the highest peak within this constrained window to find a refined onset point.
  3. Iterative Refinement: These refined points serve as "ground truth" labels to re-train the neural network. As the network learns the specific characteristics of note attacks (e.g., the ~30ms attack phase of a trumpet), it provides increasingly accurate labels for subsequent iterations.
The alignment process links score notes to acoustic events via a PDF hint.

Experimental Results

The method was evaluated using the "English Suite" for Bb Trumpet, comprising 673 notes across real and synthetic recordings. The bootstrap method demonstrated significant superiority over baseline models trained only on synthetic data:

  • Real-World Performance: On acoustic recordings, the bootstrap method reduced the onset miss rate from 15% to just 2.0% and the spurious note rate from 25% to 4.0%.
  • Timing Accuracy: The average timing error on real recordings plummeted from 35ms to 8ms. This level of precision far exceeds the 50ms resolution of the initial alignment, proving the model's ability to self-correct and refine its boundaries.

Conclusion and Future Directions

The bootstrap approach effectively solves the "cold start" problem in audio segmentation by using existing symbolic data to generate high-quality training labels automatically. While currently optimized for monophonic trumpet music, future research will explore extending this framework to polyphonic textures and a wider variety of instrumental timbres.

Summary updated

BibTeX
@inproceedings{hu2005bootstrap,
  author = {Ning Hu and Roger B. Dannenberg},
  title = {{A Bootstrap Method for Training an Accurate Audio Segmenter}},
  booktitle = {ISMIR},
  year = {2005},
}
Figure