Interpreting Glottal Flow Dynamics for Detecting Covid-19 From Voice

Soham Deshmukh, Mahmoud Al Ismail, Rita Singh

TL;DR

Researchers at Carnegie Mellon University have developed a novel way to detect COVID-19 by analyzing how the disease disrupts the physics of voice production. By comparing a patient's actual glottal flow (the airflow through the vocal folds) against a theoretical physical model of "healthy" phonation, they use a specialized "Sandwiched" attention-based neural network to identify subtle respiratory anomalies, achieving an ROC-AUC of up to 0.90.

What Problem Does This Paper Address?

COVID-19 is primarily a respiratory illness, and early studies suggested that it causes vocal fold oscillations to become asynchronous and asymmetrical. However, measuring these physical changes directly is invasive and difficult.

The challenge is that we rarely have "baseline" recordings of a person’s voice from before they got sick. Without a healthy reference point for a specific individual, it’s hard to tell if a raspy voice is due to COVID-19 or just that person's natural vocal characteristic. This paper addresses this by using a physical model of phonation as a universal "healthy" proxy, looking for the divergence between how a person actually sounds and how the physics of a healthy throat says they should sound.

Methods

The authors propose a multi-stage pipeline:

  1. Glottal Flow Estimation: They extract the Glottal Flow Waveform (GFW) from recorded speech using two methods:
  2. Inverse Filtering ($GFW_{filter}$): Estimating the raw airflow from the recorded audio signal.
  3. Physical Modeling ($GFW_{model}$): Using the ADLES algorithm and a 1-mass asymmetric body cover model to simulate what the vocal fold oscillations should look like under normal conditions.
  4. Residual Analysis: The core idea is that for a healthy person, these two waveforms should match. For a COVID-19 patient, the physical model (built on assumptions of normalcy) will fail to track the actual irregular airflow, creating a "residual" or error signal.
  5. Sandwiched 2-step Attention Pooling (S2AP): To analyze this residual, they developed a CNN architecture with a unique attention mechanism. Standard 2-step attention pools information across frequency and then time. The "Sandwiched" version adds an extra feature detector between these steps to better capture specific temporal anomalies (like phase shifts or sudden glitches in vibration) that are characteristic of COVID-19.

Datasets

The study used a clinically curated dataset from Merlin Inc. in Chile.

  • Initial Pool: 512 data instances.
  • Selected Subset: 19 recordings (10 female, 9 male) collected within 7 days of medical testing.
  • Content: Recordings of extended vowels (/a/, /i/, /u/) sampled at 8kHz.
  • Validation: 3-fold cross-validation, stratified by speaker to ensure the model wasn't just "learning" individual voices.

Key Insights and Ideas

  • The "Healthy Proxy" Strategy: Instead of needing a "before" and "after" recording of the same patient, the physical model acts as a mathematical "before" state. This bypasses a major hurdle in longitudinal medical voice analysis.
  • Vowel Sensitivity: Not all sounds are created equal. The model performed significantly better on high vowels like /i/ and /u/ than on the low vowel /a/. The authors hypothesize that the vocal fold configurations for /i/ and /u/ might make it harder for the folds to vibrate normally if they are even slightly compromised by the disease.
  • Interpretability via Attention: Because the S2AP model uses attention weights, the researchers could visualize exactly when the model detected an anomaly. They found the model specifically flags moments where the two GFWs lose synchrony (phase-locking), providing a "human-readable" explanation for the AI's decision.

Key Results

  • Model Performance: The best CNN model using the S2AP architecture achieved an ROC-AUC of 0.8520 on the general task.
  • Vowel Combinations: When focusing specifically on the combination of /i/ + /u/ vowels, the performance jumped to an ROC-AUC of 0.900.
  • Architecture Impact: Moving from standard 2-step attention to the "Sandwiched" (S2AP) version improved the ROC-AUC from 0.66 to 0.79 (without extra CNN layers), proving that the specific way the model "attends" to the residual is critical.

Limitations and Future Work

  • Sample Size: The study used a very small subset (19 speakers) of the available data to ensure clinical timing accuracy (within 7 days of testing). Larger datasets are needed to confirm these findings.
  • Vowel Hypothesis: The finding that /a/ is less effective than /i/ or /u/ is currently a conjecture based on the data; it needs validation through direct physical measurements of vocal fold movement.
  • Real-world Noise: The recordings were made on "commodity devices" (smartphones), which is good for real-world utility but introduces noise that might affect the delicate inverse filtering process.
  • Generalization: Future work could explore if these "discriminative signatures" are unique to COVID-19 or if they appear in other respiratory illnesses like the flu or pneumonia.

Summary updated

Abstract

In the pathogenesis of COVID-19, impairment of respiratory functions is often one of the key symptoms. Studies show that in these cases, voice production is also adversely affected - vocal fold oscillations are asynchronous, asymmetrical and more restricted during phonation. This paper proposes a method that analyzes the differential dynamics of the glottal flow waveform (GFW) during voice production to identify features in them that are most significant for the detection of COVID-19 from voice. Since it is hard to measure this directly in COVID-19 patients, we infer it from recorded speech signals and compare it to the GFW computed from physical model of phonation. For normal voices, the difference between the two should be minimal, since physical models are constructed to explain phonation under assumptions of normalcy. Greater differences implicate anomalies in the bio-physical factors that contribute to the correctness of the physical model, revealing their significance indirectly. Our proposed method uses a CNN-based 2-step attention model that locates anomalies in time-feature space in the difference of the two GFWs, allowing us to infer their potential as discriminative features for classification. The viability of this method is demonstrated using a clinically curated dataset of COVID-19 positive and negative subjects.
BibTeX
@inproceedings{deshmukh2021interpreting,
  author = {Soham Deshmukh and Mahmoud Al Ismail and Rita Singh},
  title = {{Interpreting Glottal Flow Dynamics for Detecting Covid-19 From Voice}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=tleUWVU65y},
}

Loading related content...

Figure