Overlapping sound event recognition using local spectrogram features with the generalised hough transform

Jonathan Dennis, Huy Dat Tran, Eng Siong Chng2012 · DOI

TL;DR

This paper introduces a novel method for recognizing overlapping sound events by treating spectrograms as images and applying the Generalised Hough Transform (GHT). By extracting local features around "interest points" and using the GHT to vote for potential sound event occurrences, the system can effectively disentangle and identify multiple simultaneous sounds, outperforming existing methods.

What Problem Does This Paper Address?

Recognizing sound events is a fundamental task in audio processing, but it becomes significantly challenging when multiple sounds occur simultaneously. Traditional methods often struggle to separate and identify these overlapping events, leading to performance degradation. While some research focuses on sound source separation, this paper tackles the problem of directly recognizing which sound events are present, even when they overlap, by leveraging visual patterns in audio spectrograms.

Methods

The core of the proposed method is the application of the Generalised Hough Transform (GHT), a technique typically used in image object recognition, to audio spectrograms. The process involves:

  1. Interest-Point Detection: Identifying key locations ("interest-points") in the spectrogram that represent significant signal structures, rather than relying on global features.
  2. Local Feature Extraction: Extracting a "plus"-shaped region of spectral information around each interest-point. These features are normalized to make them robust to variations in absolute magnitude.
  3. Codebook Clustering: For each sound class and frequency subband, local features from training data are clustered to create class-specific "codebooks."
  4. Temporal Distribution Function (TDF): For each cluster, a TDF is learned, representing the probability of finding that feature relative to the sound's onset time. This captures the temporal dynamics of the sound.
  5. Generalised Hough Transform Voting: During testing, extracted features are matched to codebook entries. The TDFs of the matched entries are then used to cast votes into a class-specific Hough accumulator, hypothesizing potential sound event onsets.
  6. Verification: Peaks in the Hough accumulator are identified as potential sound events. A verification step checks the consistency of these hypotheses against training data.

This approach allows for the detection of multiple overlapping sounds by accumulating evidence from various local features across different sound classes.

Datasets

The authors created a custom dataset for their experiments:

  • Training: 20 randomly selected isolated sound event clips from 5 classes: Horn, Bells, Bottle, Phone, and Whistle.
  • Testing: 50 samples for each of 15 overlapping clip types. These were generated by combining 2 single events from the 5 classes, overlapped by at least 50%. The test set excluded samples used for training.

The paper does not mention using publicly available standard datasets for this specific task.

Key Insights and Ideas

  • Spectrograms as Images for Sound Event Recognition: The paper successfully demonstrates that treating audio spectrograms as images and applying image processing techniques like interest-point detection and the GHT can be highly effective for sound event recognition, especially in challenging overlapping scenarios.
  • Local Features for Robustness: Moving from global features to local features around interest-points makes the recognition process more robust to variations and occlusions inherent in overlapping sounds.
  • Generalised Hough Transform for Overlap: The GHT's ability to vote for multiple hypotheses simultaneously is a key enabler for recognizing overlapping sound events, as it allows different sound classes to contribute to the accumulator independently.
  • Temporal Dynamics Capture: The use of Temporal Distribution Functions (TDFs) to represent the timing of features relative to the sound onset is crucial for distinguishing between different sound events and their temporal patterns.

Key Results

The proposed method achieved significant improvements over a baseline LogMax GMM approach:

  • Average Recognition Accuracy (TP): 94.9% for the proposed method, compared to 84.0% for the LogMax GMM. This is a 10.9% absolute improvement.
  • False Alarm Rate (FA): The proposed method achieved a significantly lower average FA of 1.8%, compared to 24.5% for the LogMax GMM.
  • Class-Specific Performance: The method showed high TP for most classes (e.g., 100% for Phone, 99.4% for Horn) with very low FAs.

The authors also note that their method is not limited to recognizing only two overlapped sound classes, unlike the baseline.

Limitations and Future Work

  • Limitations:
    • The current verification step needs improvement to handle non-uniform changes in observed sound magnitude across frequency due to distortion.
  • The current experiments do not explicitly address noisy or reverberant conditions.
  • Future Work:
    • Expand the approach to handle noisy and reverberant conditions.
  • Further improve the verification step to account for magnitude variations.
  • The authors suggest that the algorithm's parallelizable nature could lead to real-time recognition.

Summary updated

BibTeX
@misc{dennis2012overlapping,
  author = {Jonathan Dennis and Huy Dat Tran and Eng Siong Chng},
  title = {{Overlapping sound event recognition using local spectrogram features with the generalised hough transform}},
  year = {2012},
  doi = {10.21437/interspeech.2012-595},
}
Figure