A Comparison of Pooling Methods on LSTM Models for Rare Acoustic Event Classification

Chieh-Chi Kao, Ming Sun, Weiran Wang, Chao Wang

Abstract: Acoustic event classification (AEC) and acoustic event detection (AED) refer to the task of detecting whether specific target events occur in audios. As long short-term memory (LSTM) leads to state-of-the-art results in various speech related tasks, it is employed as a popular solution for AEC as well. This paper focuses on investigating the dynamics of LSTM model on AEC tasks. It includes a detailed analysis on LSTM memory retaining, and a benchmarking of nine different pooling methods on LSTM models using 1.7M generated mixture clips of multiple events with different signal-to-noise ratios. This paper focuses on understanding: 1) utterance-level classification accuracy; 2) sensitivity to event position within an utterance. The analysis is done on the dataset for the detection of rare sound events from DCASE 2017 Challenge. We find max pooling on the prediction level to perform the best among the nine pooling approaches in terms of classification accuracy and insensitivity to event position within an utterance. To authors' best knowledge, this is the first kind of such work focused on LSTM dynamics for AEC tasks.

Introduction

In Acoustic Event Classification (AEC), the primary objective is to identify the presence of specific sounds—such as a baby crying or a gunshot—within a given audio segment. Long Short-Term Memory (LSTM) networks are frequently utilized for this task due to their proficiency in handling sequential data and their efficiency on mobile hardware.

However, LSTMs face a significant challenge known as the "forgetting" problem. While designed for long-term dependencies, practical research indicates they typically maintain clear memory for only 500–1,000 time steps. In a 30-second audio clip (approximately 3,000 frames), a rare event lasting only 2 seconds might be forgotten by the time the LSTM processes the end of the sequence. This study investigates how various pooling methods—techniques for aggregating frame-level data into a single utterance-level classification—can mitigate this memory loss.

Methodology: Pooling Strategies

The researchers compared two fundamental architectural approaches to pooling across nine specific methods:

  1. Pooling on Feature: The LSTM extracts a hidden feature vector ($h_t$) for every frame. These vectors are aggregated (via Average, Max, or Attention) into a single global feature vector ($h$), which is then processed by a final dense classification layer to produce prediction $y$.
  2. Pooling on Prediction: The model applies a dense layer to every frame's feature vector to generate frame-level predictions ($y_t$). These individual predictions are then aggregated (using Max, Average, or Softmax variants) to produce the final utterance-level result $y$.
Architectural differences between pooling on features (a) and pooling on predictions (b).

Additionally, the study explored Bi-directional LSTMs (Bi-LSTMs). By processing audio in both forward and backward directions, the model can theoretically capture events at the beginning of a clip more effectively, as they remain "fresh" for the backward-running layer.

Experimental Setup

The analysis utilized the DCASE 2017 Challenge Task 2 dataset, focusing on rare sound events: baby crying, glass breaking, and gunshots. To rigorously evaluate position sensitivity, the authors generated 1.7 million synthetic 30-second clips. Short event samples were manually placed at varying timestamps (0 to 29 seconds) to observe how the model's performance fluctuated based on the event's temporal location.

Results and Discussion

The Forgetting Curve and Position Sensitivity

By testing the "Last Frame" pooling method—which relies solely on the LSTM's final state—the authors visualized a clear forgetting curve. Events occurring early in the clip were frequently missed, while recall improved as events moved closer to the end of the recording.

Recall vs. event starting position for babycry mixtures, demonstrating how different pooling methods handle the LSTM's temporal memory.

As shown in the figure above, while the LastFrame method (red) is highly sensitive to position, more robust methods like MaxPooling (green) maintain high recall across almost the entire duration of the clip. The sharp drop observed at the very end of the clips (25-30s) for several methods is attributed to annotation lag or truncation issues.

Comparative Performance

  • Top Performer: Max Pooling on Prediction (Y.MaxPooling) emerged as the most effective strategy, achieving the highest average accuracy of 91.03%.
  • Predictions vs. Features: Generally, aggregating frame-level predictions outperformed aggregating hidden features.
  • Max Pooling for Rare Events: While average pooling is common in general audio tagging, Max Pooling is superior for rare events. It functions as a "trigger"; once a strong signal for a target event is detected, that high value is preserved regardless of subsequent silence.
  • Bi-LSTM Benefits: For position-sensitive methods like Average Pooling, switching to a Bi-LSTM significantly improved worst-case recall, ensuring events were captured by at least one processing direction.

Conclusion

This study demonstrates that the choice of pooling method is critical for detecting rare acoustic events in long sequences. Max Pooling on the prediction level (Y.MaxPooling) provides the best balance of accuracy and temporal robustness. Future work may extend these findings to multi-label datasets and real-time streaming environments where bi-directional processing is not feasible.

Summary updated

Abstract

Acoustic event classification (AEC) and acoustic event detection (AED) refer to the task of detecting whether specific target events occur in audios. As long short-term memory (LSTM) leads to state-of-the-art results in various speech related tasks, it is employed as a popular solution for AEC as well. This paper focuses on investigating the dynamics of LSTM model on AEC tasks. It includes a detailed analysis on LSTM memory retaining, and a benchmarking of nine different pooling methods on LSTM models using 1.7M generated mixture clips of multiple events with different signal-to-noise ratios. This paper focuses on understanding: 1) utterance-level classification accuracy; 2) sensitivity to event position within an utterance. The analysis is done on the dataset for the detection of rare sound events from DCASE 2017 Challenge. We find max pooling on the prediction level to perform the best among the nine pooling approaches in terms of classification accuracy and insensitivity to event position within an utterance. To authors' best knowledge, this is the first kind of such work focused on LSTM dynamics for AEC tasks.
BibTeX
@inproceedings{kao2020comparison,
  author = {Chieh-Chi Kao and Ming Sun and Weiran Wang and Chao Wang},
  title = {{A Comparison of Pooling Methods on LSTM Models for Rare Acoustic Event Classification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2020},
  doi = {10.1109/icassp40776.2020.9053150},
}
Figure