Abstract
Removing background noise from acoustic observations to obtain clean signals is an important research topic regarding numerous real acoustic applications. Owing to their strong model capacity in function mapping, deep neural network-based algorithms have been successfully applied in target signal enhancement in acoustic applications. As most target signals carry semantic information encoded in a hierarchal structure in short- and long-term contexts, noise may distort such structures nonuniformly. In most deep neural network-based algorithms, such local and global effects are not explicitly considered in a modeling architecture for signal enhancement. In this article, we propose a temporal attentive pooling (TAP) mechanism combined with a conventional convolutional recurrent neural network (CRNN) model, called TAP-CRNN, which explicitly considers both global and local information for acoustic signal enhancement (ASE). In the TAP-CRNN model, we first use a convolution layer to extract local information from acoustic signals and a recurrent neural network (RNN) architecture to characterize temporal contextual information. Second, we exploit a novel attention mechanism to contextually process salient regions of noisy signals. We evaluate the proposed ASE system using an infant cry dataset. The experimental results confirm the effectiveness of the proposed TAP-CRNN, compared with related deep neural network models, and demonstrate that the proposed TAP-CRNN can more effectively reduce noise components from infant cry signals with unseen background noises at different signal-to-noise levels. We further tested the TAP-CRNN ASE system on a downstream infant cry detection (ICD) system, which determines whether a sound segment is involved in an infant cry event. Experimental results show that TAP-CRNN ASE can effectively reduce the noise components, thereby improving the performance of ICD under noisy conditions.
Introduction
Acoustic Signal Enhancement (ASE) is a critical task in audio processing, aimed at recovering a clean target signal from observations corrupted by background noise. While deep learning has significantly advanced the field, existing models often struggle with the hierarchical nature of sound, where semantic information is distributed across both short-term "local" features and long-term "global" contexts.
Standard architectures typically rely on fixed pooling operations, such as average or maximum pooling, to aggregate temporal features. However, these methods treat all time frames as equally important, failing to account for the non-uniform distortion caused by noise. This paper addresses the need for a more sophisticated feature aggregation strategy that can intelligently focus on salient signal regions while suppressing noise.
The TAP-CRNN Architecture
The proposed TAP-CRNN architecture integrates three primary components to effectively model and enhance acoustic signals:
- Convolutional Neural Network (CNN): This stage extracts local spatial-temporal features from Log Power Spectral (LPS) representations of the input audio.
- Bidirectional Long Short-Term Memory (BLSTM): A recurrent network that captures long-term temporal dependencies by processing the signal in both forward and backward directions.
- Temporal Attentive Pooling (TAP): The core innovation of this work. Instead of static pooling, the TAP block employs a dual-attention mechanism. Global Attention identifies salient regions across the entire sequence, while Local Attention refines feature extraction within these regions to ensure the most discriminative information is preserved.
The output of the TAP process is merged with the RNN's temporal summary and processed through fully connected layers to reconstruct both the enhanced acoustic spectrum and the noise spectrum independently.
Experimental Evaluation
The model was evaluated using an infant cry dataset, which presents a challenging ASE task due to the complex, non-stationary nature of the target signal. The training set comprised 7,200 utterances corrupted by six different noise types (e.g., babble, pink noise, speech) at varying Signal-to-Noise Ratios (SNRs). To assess generalization, the model was tested on an unseen set featuring noise types not present during training, such as fan noise and background music.
Furthermore, the enhanced signals were used as input for a downstream Infant Cry Detection (ICD) system, built using the public ESC-50 dataset, to measure the practical utility of the enhancement in real-world classification scenarios.
Results and Discussion
The TAP-CRNN was compared against several baseline models, including standard DNN, CNN, RNN, and CRNN architectures. Experimental results demonstrate that the TAP-CRNN consistently outperforms these baselines across multiple metrics.
Specifically, the TAP-CRNN achieved an average improvement over the standard CRNN of 20.20% in Signal Distortion Ratio (SDR), 18.43% in Signal to Artifacts Ratio (SAR), and 5.15% in Segmental SNR (SSNR). The effectiveness of the TAP mechanism in improving signal quality across different noise levels is clearly visible in the comparative performance data.
In the downstream ICD task, the application of TAP-CRNN enhancement led to a significant jump in detection accuracy, from 78.4% (without enhancement) to 89.0%. Despite the added complexity of the attention mechanism, the model remains computationally efficient, suggesting its suitability for edge device deployment.
Conclusion
This work introduces the TAP-CRNN, a novel architecture that leverages temporal attentive pooling to enhance noisy acoustic signals. By explicitly modeling global and local saliency, the system achieves superior performance in cleaning complex signals like infant cries, even in the presence of unseen noise. Future research will focus on expanding the dataset diversity, exploring multimodal enhancement, and implementing model compression techniques for low-latency applications.
Summary updated