A Compact and Discriminative Feature Based on Auditory Summary Statistics for Acoustic Scene Classification

Hongwei Song, Jiqing Han, Shiwen DengInterspeech 2018

TL;DR

This paper introduces a novel feature extraction method for acoustic scene classification (ASC) that leverages auditory summary statistics, inspired by human auditory perception. By focusing on time-averaged statistical properties of sound textures, the method achieves a highly compact yet discriminative representation, outperforming traditional handcrafted features and achieving state-of-the-art results on benchmark datasets.

What Problem Does This Paper Address?

Acoustic Scene Classification (ASC) aims to identify the environment an audio recording comes from (e.g., "park," "train station," "office"). A major challenge in ASC is finding effective features that can accurately represent the complex and often unstructured nature of environmental sounds. Traditional approaches often borrow features from speech and music processing, which may not be optimal for the temporal and spectral characteristics of everyday sounds. While some methods attempt to model the entire acoustic scene, this paper focuses on the observation that the background of an acoustic scene exhibits temporal homogeneity, akin to a "sound texture." This texture can be characterized by statistical properties rather than fine-grained temporal details, much like how the human auditory system perceives sound textures. The problem is to find a compact and discriminative feature representation that captures these statistical properties effectively.

Methods

The core of this paper is the proposed Auditory Summary Statistics (ASS) feature. The extraction process is inspired by a model of the human auditory system:

  1. Auditory Filtering and Processing:
    • Input audio is passed through a 32-channel cochlear filter bank to simulate frequency selectivity.
  2. The envelope of each subband signal is computed (using Hilbert transform and magnitude) and then compressed (raised to the power of 0.3) to mimic basilar membrane compression. These are called subband envelopes.
  3. These envelopes are further down-sampled.
  4. A bank of 20 modulation filters is applied to each subband envelope to simulate midbrain and thalamic modulation tuning effects, resulting in 32x20 modulated signals.
  5. Statistics Measurement:
    • Statistical moments (mean, variance, skew) are calculated for subband variance, envelope mean/variance/skew, and modulation power/correlation.
  6. Pairwise correlations between subband envelopes and between modulation bands are computed.
  7. Crucially, the authors select a subset of these statistics that are deemed perceptually important and less redundant, based on prior neuroscience work. This subset includes specific marginal moments and correlation statistics.
  8. These selected statistics are concatenated into a large vector, the ASS-vector.
  9. Linear Discriminant Analysis (LDA):
    • To address the high dimensionality of the ASS-vector and to maximize between-scene variance while minimizing within-scene variance (especially important given variations in recording location for the same scene), LDA is applied.
  10. This transforms the ASS-vectors into a much lower-dimensional ASS-LDA feature. The paper notes that the dimension is set to 18 for LITIS Rouen and 14 for DCASE2016.

Datasets

The proposed method was evaluated on two established acoustic scene classification datasets:

  • LITIS Rouen dataset: This is described as the largest ASC dataset. Due to its unbalanced nature, performance is reported using mean Average Precision (mAP) over 20-fold cross-validation splits.
  • DCASE2016 dataset: This dataset consists of development and evaluation sets. The authors report mean accuracy over 4-fold cross-validation on the development set and accuracy on the challenge evaluation set.

No new benchmarks were introduced; the paper focuses on evaluating its method against existing state-of-the-art approaches on these standard datasets.

Key Insights and Ideas

  • Sound Textures as Statistical Distributions: The core idea is that acoustic scenes, particularly their background components, behave like "sound textures." These textures are best characterized by time-averaged statistical properties (like variance, skew, correlations) rather than fine-grained temporal details. This is inspired by human auditory perception research.
  • Perceptual Relevance of Statistics: Not all statistical measures are equally important for perception. By carefully selecting a subset of statistics that are perceptually relevant and less redundant, the method achieves efficiency and effectiveness.
  • LDA for Discriminative Compactness: Linear Discriminant Analysis is crucial for creating a compact and discriminative feature. It effectively reduces dimensionality while preserving and enhancing the differences between acoustic scenes, even accounting for variations within the same scene due to recording conditions.
  • Segmentation is Key: Extracting features from short audio segments (e.g., 2 seconds) is vital. This ensures the temporal homogeneity assumption holds for the segment and limits the influence of rare, short-term sound events that could otherwise skew the statistics of a longer audio clip.

Key Results

  • Performance Improvement: The ASS-LDA feature significantly outperformed traditional handcrafted features on the LITIS Rouen dataset, achieving a mAP of 95.4% with a very compact 18-dimensional feature.
  • State-of-the-Art on DCASE2016: On the DCASE2016 dataset, the proposed ASS-LDA method achieved state-of-the-art performance, reaching 89.5% accuracy on the evaluation set, surpassing several other advanced methods.
  • Compactness: The ASS-LDA feature is remarkably compact (18 or 14 dimensions) compared to many other methods, making it computationally efficient and easier to combine with other features.
  • Visualization: tSNE visualizations showed that the ASS-LDA features were better clustered than the raw ASS-vectors, visually confirming that LDA successfully enhanced discriminative information.
  • Segment Length: Experiments showed that shorter segment lengths (around 2 seconds) yielded the best performance, as longer segments could violate the temporal homogeneity assumption.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while their performance is strong, it was outperformed by some feature learning-based methods on the LITIS Rouen dataset.
  • The method relies on a specific subset of statistics identified from prior neuroscience work, which might not be exhaustive for all ASC tasks.
  • Future Work:
    • The authors suggest that the compact ASS-LDA feature could be easily combined with other complementary features to further improve system performance.
  • Exploring other feature learning strategies or more advanced dimensionality reduction techniques could be potential avenues.
  • Investigating the impact of different auditory model parameters or alternative statistical measures could also be beneficial.

Summary updated

BibTeX
@inproceedings{song2018compact,
  author = {Hongwei Song and Jiqing Han and Shiwen Deng},
  title = {{A Compact and Discriminative Feature Based on Auditory Summary Statistics for Acoustic Scene Classification}},
  booktitle = {Interspeech},
  year = {2018},
}
Figure