Frame-Level Audio Feature Extraction Using AdaBoost

Norman Casagrande, Douglas Eck, Balázs KéglISMIR 2005 · OpenReview

Abstract

This paper explores the adaptation of the Viola-Jones object detection framework—originally developed for facial recognition in images—to the domain of audio signal processing. By representing audio spectrograms as visual textures, the authors employ Haar-like rectangular filters and the AdaBoost algorithm to develop a robust speech/music discriminator. The resulting system achieves a frame-level accuracy of 88%, which increases to 93.3% when temporal smoothing is applied over an 800ms window. The method's computational efficiency enables real-time performance, as demonstrated by its implementation as an audio player plugin.

Introduction

The automated classification of audio into speech and music categories is essential for various applications, including news transcription and radio monitoring. Traditional approaches often rely on complex, hand-engineered features such as pitch or cepstral coefficients. This research aims to overcome the computational overhead and latency issues of existing models by proposing a method that treats audio features as geometric patterns within a spectrogram, requiring minimal prior knowledge of signal structure.

Methods

The primary innovation lies in treating the audio spectrogram as a visual image rather than a purely mathematical signal.

  1. Haar-like Features: The system utilizes simple rectangular filters that calculate the difference in signal energy between adjacent regions. These filters capture local patterns, such as the onset of consonants or harmonic structures.
  2. Integral Images: To ensure high-speed processing, the authors use integral images, allowing the sum of any rectangular area in the spectrogram to be calculated in constant time.
  3. AdaBoost: A single filter acts as a "weak learner." AdaBoost is used to iteratively select and weight hundreds of these filters to create a strong classifier.
  4. Optimization: A gradient-following approach is implemented to navigate the vast feature space efficiently during training.
  5. Temporal Smoothing: To stabilize the output, a smoothing function integrates predictions over the previous 40 frames (approximately 800ms).
Spectrograms of speech and music with an overlaid Haar-like rectangular feature.

Datasets

The model was benchmarked using the Scheirer-Slaney (1997) database, containing 240 15-second radio extracts. Real-world testing was conducted using segments from various internet radio streams to train the version used in the real-time demonstration.

Key Results

The proposed framework demonstrates significant improvements in both accuracy and speed compared to traditional frame-level features.

  • Accuracy: The model achieved 88% accuracy on individual 20ms frames. With the addition of temporal smoothing, accuracy rose to 93.3%.
  • Comparison: This performance markedly exceeds the 63% accuracy reported for the best single frame-level feature in previous benchmark studies (Scheirer and Slaney, 1997).
  • Efficiency: The system is lightweight enough for real-time stream processing, as evidenced by its successful deployment as a graphical plugin for the Winamp audio player.
  • Representation Robustness: While various spectral representations (FFT, RASTA, Log-FFT) were tested, the simple FFT representation performed remarkably well, indicating the robustness of the AdaBoost/Haar approach.
Training and testing error rates as a function of the number of features (iterations).

Conclusion and Future Work

The study confirms that the regularities of speech and music can be effectively detected as geometric patterns. By shifting the computational burden to the training phase, the model remains highly efficient during inference. Future research will explore expanding the system to multi-class classification, including tasks such as instrument recognition and genre classification, as well as refining the feature search optimization process to find more optimal features more quickly.

Summary updated

BibTeX
@inproceedings{casagrande2005frame,
  author = {Norman Casagrande and Douglas Eck and Balázs Kégl},
  title = {{Frame-Level Audio Feature Extraction Using AdaBoost}},
  booktitle = {ISMIR},
  year = {2005},
  url = {https://openreview.net/forum?id=I1Hzf3TzlDD},
}
Figure