Improving Audio Anomalies Recognition Using Temporal Convolutional Attention Network

Qiang Huang, Thomas Hain

Abstract

Automated diagnosis of audio anomalies is essential for maintaining signal integrity in speech processing and industrial monitoring. While quality assessment tools exist, identifying the specific nature of a distortion—such as packet loss, magnitude drift, or time warping—remains challenging. We propose the Temporal Convolutional Attention Network (TCAN), which combines the large receptive fields of dilated convolutions with the dynamic feature selection of self-attention. Our model demonstrates significant accuracy gains, particularly in low-noise environments where subtle anomalies are often missed by traditional architectures.

Introduction

In high-quality audio applications like dubbing and speech recognition, "bad audio" is a persistent issue. Most existing tools provide a scalar quality score (e.g., 1-to-5 rating) but fail to diagnose the underlying cause. Identifying whether a distortion is due to electrical interference, packet loss, or time-warping is the first step toward automated correction. However, these anomalies are often sparse and buried within complex speech signals, making them difficult for standard neural networks to isolate. This paper introduces TCAN to address these diagnostic challenges.

Methodology

The TCAN architecture is built upon two primary components designed to capture long-range temporal dependencies and focus on relevant signal features:

  1. Temporal Convolutional Networks (TCN): Unlike Recurrent Neural Networks (RNNs), TCNs utilize 1D dilated convolutions. By exponentially increasing the dilation rate at each layer, the network achieves a massive receptive field, allowing it to process large windows of audio data without the vanishing gradient problems typical of LSTMs.
  2. Integrated Self-Attention: The core innovation of TCAN is the insertion of a self-attention block after every convolutional layer. This allows the model to act as a dynamic filter, focusing on specific frames where distortions occur while suppressing irrelevant background speech patterns.
The TCAN architecture integrates self-attention blocks after each 1D dilated convolution layer to refine feature extraction at multiple levels of abstraction.

Experimental Setup

The model was evaluated using the TIMIT dataset, a standard corpus of clean speech. Five synthetic distortion classes were introduced to simulate real-world anomalies:

  • Random time warping: Variable segment speeds.
  • Pooling time series: Reduced temporal resolution.
  • Dropout: Random loss of signal points.
  • Magnitude drift: Random signal value fluctuations.
  • Gaussian noise: Standard additive white noise.

These distortions were tested across six Signal-to-Noise Ratios (SNRs), ranging from 5dB (high noise) to 30dB (subtle noise).

Results and Discussion

TCAN consistently outperformed baseline models, including standard CNNs, LSTMs, and basic TCNs. The model showed a relative improvement in classification accuracy of 3% to 10% across most noise levels.

A key finding was the model's robustness at high SNRs. In "cleaner" audio (30dB), where distortions are extremely subtle, traditional models often fail. TCAN’s hierarchical attention mechanism allows it to "zoom in" on these faint signatures. As shown in the performance curves, TCAN achieves higher peak accuracy and more stable convergence compared to other architectures.

Accuracy curves at 20dB SNR demonstrate that TCAN consistently outperforms baseline models like CNN and LSTM, reaching higher accuracy levels more quickly.

Confusion matrix analysis revealed that while the model is highly effective at identifying pooling and noise, time-warping remains a challenge at high SNRs due to its similarity to natural speech variations. Furthermore, the study confirmed that increasing the dilation factor directly correlates with improved diagnostic accuracy, highlighting the importance of wide temporal context.

Conclusion

The Temporal Convolutional Attention Network provides a robust framework for diagnosing specific audio anomalies. By interleaving self-attention with dilated convolutions, the model effectively filters sparse distortions from dominant speech signals. Future work will focus on evaluating the model against real-world, non-synthetic artifacts and optimizing the architecture for real-time deployment on edge devices.

Summary updated

BibTeX
@inproceedings{huang2021improving,
  author = {Qiang Huang and Thomas Hain},
  title = {{Improving Audio Anomalies Recognition Using Temporal Convolutional Attention Network}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  doi = {10.1109/icassp39728.2021.9414611},
}
Figure