COVID-19 Disease Progression Prediction via Audio Signals: A Longitudinal Study

Ting Dang, Jing Han, Tong Xia, Dimitris Spathis, Erika Bondareva, Chloë Brown, Jagmohan Chauhan, Andreas Grammenos, Apinan Hasthanasombat, Andres Floto, Pietro Cicuta, Cecilia Mascolo

Abstract

Recent advancements have demonstrated the potential of using audio data—such as cough, breathing, and voice—for one-off COVID-19 screening. However, these methods typically fail to monitor disease progression or recovery over time. This study explores the use of longitudinal audio samples to predict COVID-19 progression using sequential deep learning. Analyzing crowdsourced data from 212 individuals over periods ranging from 5 to 385 days, we developed a tracking tool using Gated Recurrent Units (GRUs). Our model achieved an AUROC of 0.79 for detection and a high correlation (0.86) for tracking recovery trends, suggesting that longitudinal audio biomarkers are effective for remote health monitoring.

Introduction

Most AI-driven COVID-19 screening tools are designed for static, "one-off" detection. While useful for initial triage, these models do not account for the dynamic nature of the disease. Clinicians and patients require tools that can monitor whether a condition is worsening or if recovery is underway. A significant challenge in this domain is that audio characteristics vary widely between individuals; a "healthy" voice for one person might sound "congested" for another. By leveraging longitudinal data—repeated samples from the same individual—we can establish personal baselines and track the "audio dynamics" of an infection over time.

Methodology

The research team developed a deep learning pipeline specifically designed to handle sequential audio data through four key stages:

  1. Feature Extraction: Raw audio was converted into high-level mathematical embeddings using VGGish, a convolutional neural network (CNN) pretrained on large-scale audio datasets.
  2. Sequential Modeling: These embeddings were processed by Gated Recurrent Units (GRUs). Unlike standard networks, GRUs possess "memory," allowing them to capture temporal dependencies in how a person's voice or breath changes over a sequence of samples.
  3. Multitask Learning: To ensure the model focused on disease biomarkers rather than language-specific acoustic features, a multitask framework was implemented. The model was trained to simultaneously predict COVID-19 status and the speaker's language, forcing the hidden layers to ignore linguistic confounding factors.
  4. Trajectory Evaluation: The model's predicted "probability of infection" curve was compared against actual test result timelines using Dynamic Time Warping (DTW) and point-biserial correlation.

Data and Cohort

The study utilized data from the COVID-19 Sounds App (University of Cambridge), involving 212 longitudinal users (106 COVID-positive, 106 COVID-negative). Audio samples including coughs, breaths, and voice recordings were collected over 5 to 385 days. The cohort was linguistically diverse, featuring speakers of eight different languages, which necessitated the multitask learning approach to maintain model generalizability.

Key Insights and Results

A critical finding of the study was the visual and acoustic evidence of vocal cord changes during infection. In spectrograms, the researchers observed that during peak infection, the harmonics in a person's voice become "blurred," indicating a loss of vocal cord control. As the patient recovers, these harmonics become clearly separable again.

Spectrogram during peak infection (Positive) showing blurred harmonics.
Spectrogram after recovery (Negative) showing clear, separable harmonics.

By modeling relative changes in audio rather than absolute values, the system effectively uses the individual's own healthy state as a control. Key performance metrics include:

  • Superior Detection: The sequential model achieved an AUROC of 0.79, a sensitivity of 0.75, and a specificity of 0.71—a ~20% relative improvement over single-shot models.
  • Recovery Tracking: The model was particularly adept at tracking the recovery phase, showing a 0.86 correlation with actual recovery reports in a subset of participants.

Limitations and Future Work

While promising, the study is limited by a relatively small longitudinal test cohort (21 individuals) and a reliance on self-reported test results. Additionally, other respiratory infections or emotional stress can cause similar acoustic changes. Future work will focus on larger, clinically validated datasets to better distinguish COVID-19 from other conditions and to develop participant-specific models for granular telemonitoring of various respiratory diseases.

Summary updated

Abstract

Recent work has shown the potential of using audio data (eg, cough, breathing, and voice) in the screening for COVID-19. However, these approaches only focus on one-off detection and detect the infection given the current audio sample, but do not monitor disease progression in COVID-19. Limited exploration has been put forward to continuously monitor COVID-19 progression, especially recovery, through longitudinal audio data. Tracking disease progression characteristics could lead to more timely treatment. The primary objective of this study is to explore the potential of longitudinal audio samples over time for COVID-19 progression prediction and, especially, recovery trend prediction using sequential deep learning techniques. Crowdsourced respiratory audio data, including breathing, cough, and voice samples, from 212 individuals over 5-385 days were analyzed. We developed a deep learning-enabled tracking tool using gated recurrent units (GRUs) to detect COVID-19 progression by exploring the audio dynamics of the individuals' historical audio biomarkers. The investigation comprised 2 parts: (1) COVID-19 detection in terms of positive and negative (healthy) tests, and (2) longitudinal disease progression prediction over time in terms of probability of positive tests. The strong performance for COVID-19 detection, yielding an AUROC of 0.79, a sensitivity of 0.75, and a specificity of 0.71 supported the effectiveness of the approach compared to methods that do not leverage longitudinal dynamics. We further examined the predicted disease progression trajectory, displaying high consistency with test results with a correlation of 0.75 in the test cohort and 0.86 in a subset of the test cohort who reported recovery. Our findings suggest that monitoring COVID-19 evolution via longitudinal audio data has potential in the tracking of individuals' disease progression and recovery.
BibTeX
@article{dang2022covid,
  author = {Ting Dang and Jing Han and Tong Xia and Dimitris Spathis and Erika Bondareva and Chloë Brown and Jagmohan Chauhan and Andreas Grammenos and Apinan Hasthanasombat and Andres Floto and Pietro Cicuta and Cecilia Mascolo},
  title = {{COVID-19 Disease Progression Prediction via Audio Signals: A Longitudinal Study}},
  journal = {Journal of Medical Internet Research},
  year = {2022},
  doi = {10.2196/37004},
}
Figure