UFRC: A Unified Framework for Reliable COVID-19 Detection on Crowdsourced Cough Audio

Jiangeng Chang, Yucheng Ruan, Shaoze Cui, John Soong Tshon Yit, Mengling Feng

Introduction

The COVID-19 pandemic underscored the urgent need for rapid, scalable, and non-invasive screening tools. While Reverse Transcription Polymerase Chain Reaction (RT-PCR) remains the diagnostic gold standard, its high cost, logistical requirements, and time-intensive nature often limit its accessibility, particularly in rural or resource-constrained environments.

Recent research has turned toward acoustic biomarkers—specifically cough sounds—as a potential solution. Respiratory infections often alter the acoustic properties of a cough in ways that are detectable by machine learning models. However, deploying these models in the real world presents significant challenges. Crowdsourced audio data is typically highly imbalanced, with far more healthy samples than infected ones, and varies wildly in recording quality. The Unified Framework for Reliable COVID-19 Detection (UFRC) was developed to address these hurdles, providing a robust pipeline that not only identifies infection but also assesses its own confidence.

The UFRC Framework

The UFRC framework integrates five distinct components designed to transform raw, noisy audio into a reliable diagnostic prediction:

1. Gaussian Noise-based Data Augmentation

Crowdsourced datasets often suffer from a lack of positive (COVID-19) samples. To improve model generalization and prevent overfitting on the limited positive cases, the framework employs Gaussian noise augmentation. By adding random noise to the raw audio signals, the system creates synthetic variations of the training data, forcing the model to learn more robust features rather than memorizing specific recording artifacts.

2. Transfer Learning via ResNet-50

One of the framework's most effective strategies is the use of transfer learning from the computer vision domain. Audio signals are converted into Mel-spectrograms—visual representations of sound frequencies over time. The researchers then utilized a ResNet-50 architecture pre-trained on ImageNet. Despite being trained on natural images (like animals and objects), the pre-trained weights provide a sophisticated feature-extraction foundation that significantly outperforms models trained from scratch on audio data.

3. Cost-sensitive Focal Loss

Standard cross-entropy loss functions often struggle with imbalanced datasets, as they can be dominated by the majority class (healthy samples). UFRC implements Focal Loss, which applies a modulating factor to the loss function. This down-weights the loss contributed by "easy" examples and focuses the model's learning on "hard" examples—specifically the rare COVID-positive samples that are critical for detection.

4. Deep Ensemble Learning

To ensure stability and accuracy, UFRC does not rely on a single model. Instead, it employs an ensemble of multiple identical architectures trained with different random initializations and data shuffling. By averaging the predictions across the ensemble, the framework reduces the variance associated with individual models and produces a more generalized final score.

5. Uncertainty Estimation

For a diagnostic tool to be clinically useful, it must recognize its own limitations. UFRC measures the disagreement between the models within the ensemble to estimate predictive uncertainty. When the models in the ensemble provide widely varying scores, the system flags the prediction as "high uncertainty," suggesting that the case should be triaged for manual review by a medical professional rather than relying solely on the automated output.

Experimental Results

The framework was evaluated using the DiCOVA2021 challenge dataset, which consists of 822 training samples and 218 validation samples. The dataset is characterized by a severe class imbalance (approximately 15:1 ratio of healthy to COVID-positive cases).

Performance Benchmarks

UFRC achieved a high AUC-ROC of 85.43%, significantly outperforming traditional machine learning baselines:

  • Random Forest: 67.59%
  • Support Vector Machine (SVM): 69.91%
  • Multilayer Perceptron (MLP): 61.97%

The Impact of Pre-training

The study confirmed the vital role of transfer learning. In a direct comparison, the ImageNet-initialized model achieved a 64.50% AUC, whereas a model with random initialization performed at 43.48%—worse than a random guess. This suggests that the low-level features learned from vision datasets (edges, textures, patterns) are highly transferable to the patterns found in audio spectrograms.

Reliability and Selective Prediction

The inclusion of uncertainty estimation proved critical for robustness. The system demonstrated higher accuracy (91%) on cases where it reported low uncertainty, compared to 86% on high-uncertainty cases. This capability allows the framework to function as a reliable first-line filter in a broader diagnostic pipeline.

Conclusion and Future Directions

The UFRC framework demonstrates that high-performance COVID-19 screening is possible using nothing more than a smartphone-recorded cough. By combining transfer learning with ensemble-based uncertainty estimation, the framework bridges the gap between experimental machine learning and reliable clinical utility.

While the results are promising, the researchers note that the current dataset is relatively small. Future work will focus on expanding the framework to detect other respiratory conditions—such as asthma, pneumonia, and influenza—and validating the model on larger, more geographically diverse datasets to ensure global generalizability.

Summary updated

BibTeX
@inproceedings{chang2022ufrc,
  author = {Jiangeng Chang and Yucheng Ruan and Shaoze Cui and John Soong Tshon Yit and Mengling Feng},
  title = {{UFRC: A Unified Framework for Reliable COVID-19 Detection on Crowdsourced Cough Audio}},
  booktitle = {EMBC},
  year = {2022},
  doi = {10.1109/embc48229.2022.9871615},
}
Figure