Pooling Robust Shift-Invariant Sparse Representations of Acoustic Signals

Po-Sen Huang, Jianchao Yang, Mark Hasegawa-Johnson, Feng Liang, Thomas S. HuangInterspeech 2012 · DOI

Abstract

Traditional audio feature extraction methods like MFCCs are often sensitive to temporal shifts and noise due to their block-based nature. This paper proposes a novel approach combining sparse time-relative coding (spikegrams) with temporal pyramid pooling. By decomposing signals into sparse spikes using gammatone kernels and aggregating them through a hierarchical pooling scheme, the authors achieve a representation that is both shift-invariant and robust to noise. Evaluation on the UPC Acoustic Event Dataset demonstrates an absolute improvement in classification accuracy of up to 12% over traditional baselines.

Introduction

The field of acoustic event detection has long relied on features derived from short-time Fourier transforms, such as Mel-frequency cepstral coefficients (MFCCs) and Perceptual Linear Prediction (PLP). While effective for stationary signals, these "block-based" methods struggle with non-stationary, transient sounds like clicks or impacts. A primary drawback is their lack of shift-invariance; a minor temporal shift in the input signal can lead to significantly different feature vectors, complicating the task for classifiers. Furthermore, the fixed windowing process often smears fine-grained temporal details across adjacent frames.

Methodology

The proposed framework addresses these issues by adapting techniques from computer vision—specifically sparse coding and spatial pyramid pooling—to the audio domain.

Sparse Time-Relative Coding (Spikegrams)

Instead of analyzing fixed windows, the signal is decomposed into a sparse set of atomic components. The authors utilize a dictionary of 64 gammatone kernels, which are designed to mimic the frequency response of the human cochlea. The decomposition is performed using the Matching Pursuit algorithm, an iterative process that finds the best-matching kernel at each step and subtracts it from the signal.

Signal and Residual: The original acoustic signal (blue) and the minimal residual (red) after sparse decomposition.

The result is a "spikegram," where each "spike" is defined by the kernel index, its magnitude, and its precise temporal location. This representation is inherently more precise than a spectrogram and naturally captures the transient nature of acoustic events.

Spikegram: A sparse representation of the acoustic signal across 64 kernels over time.

Temporal Pyramid Pooling

To convert the variable number of spikes into a fixed-length feature vector suitable for standard classifiers, the authors introduce Temporal Pyramid Pooling. Inspired by spatial pyramids in image processing, the signal is divided into a hierarchy of temporal grids (e.g., level 0 is the whole signal, level 1 is two halves, etc.). Within each grid cell, spike magnitudes are aggregated using a pooling function.

The study explored several pooling functions, including average, max, and bio-inspired non-linear compressions like magnitude and cuberoot pooling. Interestingly, while max pooling is dominant in computer vision, it performed poorly for audio spikes. Instead, magnitude and cuberoot pooling—which approximate human perception of loudness—yielded the best results, suggesting that the total energy distribution within a temporal region is more discriminative for acoustic events.

Experimental Evaluation

The method was validated using the Acoustic Event Dataset from Universitat Politècnica de Catalunya (UPC), featuring 11 classes of office-environment sounds (e.g., door slams, typing, coughing). The dataset includes approximately 90 instances per class across six recording sessions. To evaluate robustness, Gaussian white noise was added to the clean recordings at signal-to-noise ratios (SNR) of 10dB, 20dB, and 30dB. Classification was performed using a Support Vector Machine (SVM) with an intersection kernel.

Results and Discussion

The temporal pyramid pooling approach significantly outperformed traditional PLP-based Hidden Markov Models (HMMs). The proposed method achieved an absolute error reduction of 8% to 12% across various conditions. In relative terms, this represents a 14% to 34% improvement over the baseline.

One of the most striking findings was the system's robustness to noise. At a challenging 10dB SNR, the cuberoot pooling method maintained an accuracy of 43.40%, whereas the PLP-HMM baseline dropped to 28.05%. The hierarchical nature of the pyramid pooling allows the model to capture both the global "gist" of the sound and its local temporal structure, providing the desired shift-invariance.

Conclusion

By moving away from fixed-window analysis toward a sparse, spike-based representation combined with hierarchical pooling, this research provides a robust framework for acoustic event classification. The success of bio-inspired pooling functions highlights the value of incorporating auditory processing principles into machine learning features. Future work will explore the integration of these sparse representations into deep learning architectures and their performance on larger, more diverse acoustic datasets.

Summary updated

BibTeX
@inproceedings{huang2012pooling,
  author = {Po-Sen Huang and Jianchao Yang and Mark Hasegawa-Johnson and Feng Liang and Thomas S. Huang},
  title = {{Pooling Robust Shift-Invariant Sparse Representations of Acoustic Signals}},
  booktitle = {Interspeech},
  year = {2012},
  doi = {10.21437/interspeech.2012-652},
}
Figure