A comparative study in automatic recognition of broadcast audio

Stavros Ntalampiras, Nikos FakotakisInterspeech 2008

TL;DR

This paper introduces and compares three distinct feature sets for automatically distinguishing between speech and music in broadcast audio. By combining the strengths of Mel-frequency cepstral coefficients (MFCCs), MPEG-7 Audio Spectrum Projection (ASP), and Discrete Wavelet Transform (DWT), the authors achieve a remarkable 98.5% average recognition rate, paving the way for more efficient audio analysis and content management.

What Problem Does This Paper Address?

In our increasingly audio-rich digital world, manually annotating vast amounts of audio content is a monumental task. This paper tackles the challenge of automatically classifying broadcast audio into speech and music. This capability is crucial for numerous applications, such as organizing large audio archives, enabling automatic news transcription systems to focus solely on spoken content, and even creating intelligent car radio systems that can automatically switch stations to find music. The core problem is to find a way to efficiently and accurately distinguish between these two broad audio categories using computational methods.

Methods

The authors explore three distinct feature extraction methodologies:

  • Mel filterbank-based features: This approach utilizes Mel-scale filterbanks to process the audio signal, emphasizing frequencies important for human perception. A key innovation here is the use of Principal Component Analysis (PCA) to reduce the dimensionality of the extracted features, transforming them into a new coordinate system based on their inherent relationships. This is presented as a data-driven alternative to traditional Discrete Cosine Transform (DCT).
  • Audio Spectrum Projection (ASP): This descriptor, part of the MPEG-7 standard, represents audio signals by projecting their spectrum onto a low-dimensional feature space using decorrelated basis functions. The process involves computing the Audio Spectrum Envelope (ASE) and then projecting it using a PCA-derived basis.
  • DWT-based sound parameters: Leveraging the Discrete Wavelet Transform (DWT), this method excels at analyzing non-stationary signals. The authors focus on the approximation coefficients (low-frequency information) and extract six statistical measurements (mean, variance, min, max, standard deviation, median) from these coefficients. PCA is again employed to further reduce dimensionality and improve class separation.

For classification, the researchers employ Hidden Markov Models (HMMs), a standard technique for modeling sequential data. They train ergodic HMMs with 3 states and 128 Gaussian mixture components for each class. A novel aspect is the proposed fusion scheme, where ASP is favored for music recognition and DWT for speech, with a majority voting mechanism to combine decisions and handle differing frame sizes.

Datasets

The experiments were conducted on a custom-built dataset comprising 12 hours of manually annotated BBC and CNN online radio recordings. Silent segments were removed using a statistical model. Approximately 3 hours were used for training, and the remaining 9 hours for testing. The corpus is characterized by significant diversity, including seven different speakers and six musical genres (pop, rock, jazz, classical, soul, and country). Care was taken to ensure no overlap between training and testing segments from the same source.

Key Insights and Ideas

  • PCA as a data-driven feature enhancer: The paper highlights the effectiveness of PCA not just for dimensionality reduction but also for creating new, more discriminative feature spaces tailored to the specific data.
  • Complementary strengths of feature sets: While Mel filterbank features showed the best overall performance individually, the fusion strategy revealed that ASP is particularly adept at recognizing music, and DWT excels at speech recognition. This suggests that different feature sets capture distinct aspects of audio signals.
  • Fusion for improved accuracy: Combining the outputs of different feature extractors and classifiers through a majority voting scheme significantly boosted the overall recognition rate, demonstrating the power of ensemble methods.
  • Real-time potential: The authors emphasize the low complexity of their proposed approach, suggesting its suitability for real-time broadcast audio analysis.

Key Results

The individual feature sets achieved the following average recognition rates:

  • Mel filterbank: 94.7%
  • ASP: 94.35%
  • DWT: 86.55%

However, the proposed fusion scheme, combining ASP for music and DWT for speech with a majority voting mechanism, achieved a significantly higher average recognition rate of 98.5%. This breakdown further shows 99% accuracy for speech and 98% for music.

Limitations and Future Work

The authors acknowledge that the Mel filterbank-based feature set, while performing best individually, did not yield the best results for either of the partial classification tasks (speech or music). They also note that ASP was designed to better capture generic sound properties, which might explain its strong performance on music.

Future work suggested includes exploring a non-redundant fusion of all the feature sets. The current fusion scheme is described as a "simple majority voting procedure," implying that more sophisticated fusion techniques could potentially yield further improvements.

Summary updated

BibTeX
@inproceedings{ntalampiras2008comparative,
  author = {Stavros Ntalampiras and Nikos Fakotakis},
  title = {{A comparative study in automatic recognition of broadcast audio}},
  booktitle = {Interspeech},
  year = {2008},
}
Figure