A Temporal-oriented Broadcast ResNet for COVID-19 Detection

Xin Jing, Shuo Liu, Emilia Parada‐Cabaleiro, Andreas Triantafyllopoulos, Meishu Song, Zijiang Yang, Björn W. Schuller

TL;DR

This paper introduces TorNet, a novel deep learning architecture for detecting COVID-19 from cough sounds. TorNet leverages a new "Alternating Broadcast Block" to efficiently capture temporal audio features with significantly fewer parameters than traditional recurrent networks, achieving competitive accuracy on a challenging benchmark.

What Problem Does This Paper Address?

The ongoing COVID-19 pandemic necessitates rapid and accessible diagnostic tools. While traditional testing methods exist, they can be limited by equipment availability and can paradoxically increase transmission risk in crowded testing environments. Audio-based detection, specifically analyzing coughs and breathing patterns, offers a promising avenue for a fast, low-cost, and non-invasive pre-testing method. However, effectively modeling the temporal dynamics of audio signals for accurate classification is challenging. Existing deep learning approaches, such as Recurrent Neural Networks (RNNs) and Transformers, can be computationally expensive and prone to overfitting, making them less ideal for real-time, on-device applications. This paper aims to bridge this gap by proposing an efficient deep learning architecture that excels at capturing temporal audio features for COVID-19 detection.

Methods

The core innovation of this paper is the Temporal-oriented ResNet (TorNet) architecture, built upon the principles of EfficientNet and residual learning. TorNet introduces a novel Alternating Broadcast Block (AB Block). This block is designed to efficiently extract temporal features from audio spectrograms.

Key components of the AB Block include:

  • Broadcast Residual Blocks (BC ResBlocks): These blocks are adapted from previous work and are designed to remap temporal information to the original feature map. They incorporate frequency-wise depth-wise convolutions and a unique broadcasting operation that expands temporal features.
  • Frequency Averaging: To emphasize temporal features, the AB Block averages 2D features in the frequency dimension before applying temporal convolutions.
  • Instance Normalization (IN): The authors employ Instance Normalization, particularly on the frequency dimension, to improve domain generalization and make the model more robust to variations in audio data. This is a key differentiator from standard batch normalization.
  • Convolutional Layer: A final convolution layer within the AB Block helps to capture global temporal information and compensate for potential information loss in the frequency dimension during averaging.

TorNet itself is structured into multiple stages, each containing AB Blocks. The architecture progressively downsamples the feature maps while increasing the channel depth, allowing it to learn hierarchical representations. The use of residual connections and the AB Block's design contribute to a smaller model size and higher computational efficiency compared to RNNs and Transformers.

Datasets

The primary dataset used for evaluation is from the INTERSPEECH 2021 Computational Paralinguistics Challenge (CCS) COVID-19 cough sub-challenge. This dataset consists of 929 cough recordings from 397 participants, labeled as either COVID-19 positive or negative. The recordings were resampled to 16 kHz and converted to mono/16 bits. The official training, validation, and test sets provided by the challenge were utilized.

Key Insights and Ideas

  • Temporal Features are Crucial, but Efficiency Matters: The paper strongly reinforces the idea that temporal information is vital for accurate audio classification, especially for tasks like COVID-19 detection from coughs. However, it highlights that traditional methods for capturing these features (RNNs, Transformers) come with a significant computational cost.
  • Broadcasting for Temporal Efficiency: The novel AB Block's "broadcasting" mechanism is a clever way to inject temporal information into feature maps without the sequential processing overhead of RNNs. This allows for a more parallelizable and efficient architecture.
  • Instance Normalization for Robustness: The application of Instance Normalization, particularly in the frequency domain, proves to be a valuable technique for improving the model's generalization capabilities, making it less sensitive to variations in the audio data.
  • Intermediate Layer Temporal Modeling is Superior: The authors demonstrate that integrating temporal feature extraction within the intermediate layers of a CNN (as in TorNet) is more effective than simply adding recurrent layers at the end of a network. This suggests a more holistic approach to temporal modeling.

Key Results

  • Competitive Accuracy: TorNet achieved an Unweighted Average Recall (UAR) of 72.2% on the INTERSPEECH 2021 CCS COVID-19 cough sub-challenge test set. This result is competitive with state-of-the-art methods.
  • High Computational Efficiency: TorNet boasts a significantly smaller model size (4.46 million parameters) compared to other high-performing methods like CNN14 (79.67 million parameters) and fusion-based approaches. This translates to lower computational requirements for training and inference.
  • Ablation Study Highlights Key Components:
    • Removing the last convolution layer in the AB Block significantly reduced UAR to 65.5%, underscoring its importance for capturing global temporal context.
  • Adding Instance Normalization improved UAR from 70.2% to 72.2%, demonstrating its benefit for domain generalization.
  • Comparison to Baselines: TorNet outperformed the official baseline (End2You at 64.7%) and other ResNet-based variants with added LSTM or Transformer layers, indicating the superiority of its integrated temporal modeling approach.

Limitations and Future Work

Limitations:

  • While TorNet achieves competitive results, it still lags behind some approaches that utilize large-scale transfer learning (e.g., Casanova et al.'s CNN14 at 75.9% UAR). However, these methods often involve much higher computational costs and longer training times.
  • The paper notes that data augmentation techniques like mixup and Spec augmentation did not yield noticeable performance gains for this specific task, suggesting potential limitations in their applicability or the dataset's characteristics.

Future Work:

The authors suggest that TorNet's architecture is promising and could be applied to other domains, such as:

  • Speech emotion recognition
  • Acoustic scene classification

Summary updated

Abstract

Detecting COVID-19 from audio signals, such as breathing and coughing, can be used as a fast and efficient pre-testing method to reduce the virus transmission. Due to the promising results of deep learning networks in modelling time sequences, we present a temporal-oriented broadcasting residual learning method that achieves efficient computation and high accuracy with a small model size. Based on the EfficientNet architecture, our novel network, named Temporaloriented ResNet (TorNet), constitutes of a broadcasting learning block. The network obtains useful audio-temporal features and higher level embeddings effectively with much less computation than Recurrent Neural Networks (RNNs), typically used to model temporal information. TorNet achieves 72.2% Unweighted Average Recall (UAR) on the INTERPSEECH 2021 Computational Paralinguistics Challenge COVID-19 cough Sub-Challenge, by this showing competitive results with a higher computational efficiency than other state-of-the-art alternatives.
BibTeX
@inproceedings{jing2022temporal,
  author = {Xin Jing and Shuo Liu and Emilia Parada‐Cabaleiro and Andreas Triantafyllopoulos and Meishu Song and Zijiang Yang and Björn W. Schuller},
  title = {{A Temporal-oriented Broadcast ResNet for COVID-19 Detection}},
  booktitle = {2022},
  year = {2022},
  doi = {10.1109/bhi56158.2022.9926930},
}
Figure