Pitched Instrument Onset Detection based on Auditory Spectra

Emmanouil Benetos, Andre Holzapfel, Yannis StylianouISMIR 2009 · OpenReview

Abstract

Musical onset detection is a fundamental task in music information retrieval, yet it remains challenging for pitched instruments with gradual "soft" attacks. This paper proposes a bio-inspired detection framework that utilizes auditory spectra—modeled after human cochlear and cortical processing—rather than standard linear Fourier transforms. By extracting spectral flux and group delay features from these auditory representations, the system achieves superior performance on a diverse dataset of 11 pitched instruments. Experimental results demonstrate that Auditory Spectral Flux (AUD_SF) provides a 2% improvement in F-measure over traditional methods, reaching an overall accuracy of 75.9%.

Introduction

Identifying the precise start of a musical note, or onset detection, is a prerequisite for tasks such as automatic transcription and beat tracking. While percussive onsets are characterized by sharp energy spikes, pitched instruments like the violin or flute often exhibit gradual energy build-ups, making their onsets harder to pinpoint. Standard detection algorithms typically rely on the Discrete Fourier Transform (DFT). However, the human auditory system processes sound non-linearly and logarithmically. This research explores whether modeling these biological mechanisms can lead to more robust onset detection for pitched instruments.

Methodology

The proposed system replaces the standard DFT pipeline with a two-stage auditory model. The first stage, the Cochlear Stage, mimics the spectral estimation of the inner ear. The second stage, the Cortical Stage, simulates how the primary auditory cortex analyzes these signals. This process generates an "auditory spectrogram" with 128 log-frequency bins, aligning with the tonotopic organization of the ear.

Block diagram of the proposed auditory-based onset detection system.

From this representation, two primary features are derived:

  1. Auditory Spectral Flux (AUD_SF): Measures sudden positive energy changes across auditory bins to identify new energy appearing in the signal.
  2. Auditory Group Delay (AUD_GRD): A phase-based feature that helps locate impulses by analyzing the derivative of phase over frequency.

The processing pipeline concludes with signal smoothing, normalization, adaptive thresholding, and peak picking to identify onset times.

Experimental Setup and Results

The system was evaluated using a dataset of 57 monophonic recordings covering 11 instrument types, including Western (e.g., Cello, Piano, Saxophone) and Middle-Eastern instruments (e.g., Ney, Tanbur). The dataset contains 1,829 manually annotated onsets.

Performance comparison across different instrument categories: (a) All Instruments, (b) Bowed Strings, (c) Pitched Percussive, and (d) Wind Instruments.

The auditory-based features consistently outperformed traditional DFT-based counterparts. Auditory Spectral Flux (AUD_SF) emerged as the top performer with an F-measure of 75.9%, representing a 2% absolute improvement over DFT-based spectral flux.

Key Insights

  • Bio-mimicry as a Filter: The auditory model naturally suppresses noise outside the range of human perception, effectively pre-cleaning the signal.
  • Logarithmic Advantage: The log-frequency bins of the auditory model better match the harmonic structure of music compared to linear DFT bins.
  • Instrument Specificity: While AUD_SF is generally more robust, Auditory Group Delay (AUD_GRD) showed exceptional precision for bowed strings. Conversely, for pitched percussive instruments like the piano, traditional DFT methods still hold a slight edge due to the high-frequency transients that the auditory model may over-smooth.

Conclusion

This study demonstrates that incorporating biological hearing models into MIR tasks can significantly enhance performance for complex signals like pitched instrument onsets. Future work will explore the fusion of energy and phase features and extend the model to polyphonic audio environments.

Summary updated

BibTeX
@inproceedings{benetos2009pitched,
  author = {Emmanouil Benetos and Andre Holzapfel and Yannis Stylianou},
  title = {{Pitched Instrument Onset Detection based on Auditory Spectra}},
  booktitle = {ISMIR},
  year = {2009},
  url = {https://openreview.net/forum?id=UCaJ2AjCov},
}
Figure