COVID-Net Assistant: A Deep Learning-Driven Virtual Assistant for COVID-19 Symptom Prediction and Recommendation

Pengyuan Shi, Yuetong Wang, Saad Abbasi, Alexander WongarXiv 2022 · DOI · arXiv

Abstract

COVID-Net Assistant is a deep learning-driven virtual assistant designed to provide low-cost, accessible COVID-19 screening by analyzing cough recordings. By utilizing machine-driven architecture design and high-quality PCR-verified datasets, the system achieves an impressive Area Under the Curve (AUC) of over 0.95 while remaining efficient enough to run on mobile devices and single-board computers. This paper details the development pipeline, architectural exploration, and performance benchmarks of the assistant.

Introduction

While RT-PCR testing remains the gold standard for COVID-19 detection, its resource-intensive nature and slow turnaround times (often 2–3 days) create significant barriers to rapid screening, particularly in developing regions. Rapid antigen tests offer a faster alternative but are not always accessible. There is a critical need for an inexpensive, non-invasive "pre-screening" tool to help prioritize medical resources.

COVID-19 is known to affect respiratory patterns, yet the subtle differences in cough sounds are often imperceptible to the human ear. Machine learning offers a solution, but existing models are often hampered by noisy, self-reported datasets. COVID-Net Assistant addresses these challenges through a robust processing pipeline and optimized neural architectures.

Methodology

The COVID-Net Assistant pipeline transforms raw audio into a format suitable for deep learning analysis, following a structured workflow from recording to recommendation.

The end-to-end workflow of the COVID-Net Assistant pipeline, from cough recording to medical advice.

Signal Processing and Augmentation

Raw audio (48kHz) is converted into Mel-frequency cepstral coefficients (MFCCs). The researchers utilized 32 MFCC coefficients and 32 mel bands to generate a 32 × 328 × 1 "image-like" representation of the sound. To enhance model robustness and prevent overfitting on limited data, various augmentation techniques were applied, including random trimming, shifting, Gaussian noise, and pitch shifting.

Machine-Driven Design Exploration

Rather than manual architecture tuning, the authors employed "generative synthesis." This automated process explores macro and micro-architecture designs to find the optimal balance between accuracy and efficiency. Three primary "seed" designs were evaluated:

  • Standard CNNs: Optimized for predictive performance.
  • Res-CNNs: Utilizing residual blocks to facilitate deeper architectures.
  • DW-CNNs: Utilizing depth-wise separable convolutions for extreme computational efficiency on edge devices.

Dataset Analysis

The study utilized the Covid19-Cough benchmark dataset (1,322 samples). A key contribution of this work is the distinction between two data splits:

  1. All-Data: Includes 681 positive samples, many of which are self-reported.
  2. Verified-Only: Includes only the 381 positive samples strictly confirmed via PCR tests.

The researchers found that models trained on PCR-verified data performed significantly better (over 0.1 AUC higher) than those trained on self-reported data. This highlights the danger of "citizen science" datasets lacking clinical verification, which can introduce noise that hinders model development.

Results and Discussion

The experimental results demonstrate that machine-driven design can produce highly efficient models without sacrificing predictive power.

Predictive performance (AUC) and computational complexity (Params, FLOPs) across different architectural variants.

Predictive Power

The best-performing model, a standard CNN trained on the Verified-Only split, achieved a peak AUC of 0.9508. Interestingly, while depth-wise separable convolutions (DW-CNN) are typically the standard for mobile efficiency, the machine-optimized standard CNN achieved the highest accuracy on verified data.

Computational Efficiency and Latency

The DW-CNN-S variant demonstrated remarkable efficiency, featuring only 3.5K parameters and requiring 3.9 million FLOPs. In terms of real-world deployment:

  • Workstation Latency: Inference as low as 1.72ms.
  • Edge Latency: On a single-board computer (SBC), the DW-CNN-S model achieved a latency of 4.94ms, proving its viability for real-time deployment on low-power devices.

Conclusion and Future Work

COVID-Net Assistant demonstrates the feasibility of high-fidelity COVID-19 pre-screening via mobile devices. The study underscores the importance of high-quality, clinically verified data in training medical AI.

Future research will focus on semi-supervised learning to leverage larger unlabeled datasets, and Explainable AI (XAI) to visualize the specific acoustic patterns indicative of infection. These advancements will further refine the tool's accuracy and provide deeper biological insights into respiratory symptoms.

Summary updated

Abstract

As the COVID-19 pandemic continues to put a significant burden on healthcare systems worldwide, there has been growing interest in finding inexpensive symptom pre-screening and recommendation methods to assist in efficiently using available medical resources such as PCR tests. In this study, we introduce the design of COVID-Net Assistant, an efficient virtual assistant designed to provide symptom prediction and recommendations for COVID-19 by analyzing users' cough recordings through deep convolutional neural networks. We explore a variety of highly customized, lightweight convolutional neural network architectures generated via machine-driven design exploration (which we refer to as COVID-Net Assistant neural networks) on the Covid19-Cough benchmark dataset. The Covid19-Cough dataset comprises 682 cough recordings from a COVID-19 positive cohort and 642 from a COVID-19 negative cohort. Among the 682 cough recordings labeled positive, 382 recordings were verified by PCR test. Our experimental results show promising, with the COVID-Net Assistant neural networks demonstrating robust predictive performance, achieving AUC scores of over 0.93, with the best score over 0.95 while being fast and efficient in inference. The COVID-Net Assistant models are made available in an open source manner through the COVID-Net open initiative and, while not a production-ready solution, we hope their availability acts as a good resource for clinical scientists, machine learning researchers, as well as citizen scientists to develop innovative solutions.
BibTeX
@misc{shi2022covid,
  author = {Pengyuan Shi and Yuetong Wang and Saad Abbasi and Alexander Wong},
  title = {{COVID-Net Assistant: A Deep Learning-Driven Virtual Assistant for COVID-19 Symptom Prediction and Recommendation}},
  year = {2022},
  eprint = {2211.11944},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2211.11944},
}
Figure