Machine learning analysis to automatically measure response time of pharyngeal swallowing reflex in videofluoroscopic swallowing study

Jong Taek Lee, Eunhee Park, Jong-Moon Hwang, Tae‐Du Jung, Donghwi ParkScientific 2020 · DOI

TL;DR

This paper introduces a novel machine learning framework to automatically measure the response time of the pharyngeal swallowing reflex from videofluoroscopic swallowing study (VFSS) videos. This addresses the labor-intensive and error-prone nature of manual measurement, offering a clinically useful tool for diagnosing dysphagia and guiding rehabilitation strategies.

What Problem Does This Paper Address?

Dysphagia, or difficulty swallowing, is a significant clinical challenge, particularly in patients with neurological conditions like stroke or Parkinson's disease. A critical indicator of dysphagia is an abnormal (delayed or absent) pharyngeal swallowing reflex, which can lead to aspiration and pneumonia. Videofluoroscopic swallowing study (VFSS) is the gold standard for visualizing this process. However, manually measuring the precise response time of the pharyngeal swallowing reflex from VFSS videos is extremely difficult and time-consuming, especially for inexperienced clinicians. This manual process suffers from poor inter-rater reliability, leading to inconsistent diagnoses and treatment plans. The paper aims to bridge this gap by developing an automated, accurate, and reliable method for measuring this crucial temporal parameter.

Methods

The core of this work is a machine learning framework built upon the Inflated 3D Convolutional Network (I3D) architecture, specifically leveraging a pre-trained Inception-V1 model. This choice is motivated by I3D's proven effectiveness in action recognition and prior success in VFSS analysis for pharyngeal phase detection.

A key challenge addressed is the very short duration of the swallowing reflex (often less than a second), which is significantly shorter than the typical input frame length for 3D CNNs. To overcome this, the researchers implemented a novel data augmentation strategy. Instead of simply classifying "swallowing reflex" vs. "not swallowing reflex," they defined three classes: "before," "during," and "after" the swallowing reflex. This allowed them to generate multiple training samples from a single annotated video clip, effectively expanding the dataset and enabling the model to learn from shorter temporal intervals.

The model was trained using a fivefold cross-validation approach to ensure generalization. Evaluation metrics included the detection F-1 score and the time error for both the starting and ending points of the reflex.

Datasets

The study utilized a dataset of 207 pharyngeal swallowing event clips extracted from raw VFSS videos. These clips were collected from 27 participants (22-84 years old) who reported subjective swallowing difficulties. The participants included individuals with central nervous system diseases, neuromuscular diseases, and a subset of healthy controls. The ground-truth annotations for the starting and ending points of the pharyngeal swallowing reflex were meticulously labeled by two expert physiatrists.

Key Insights and Ideas

  • Bridging the Temporal Gap: The most significant insight is the successful adaptation of 3D CNNs, typically designed for longer video sequences, to accurately detect very short, rapid events like the pharyngeal swallowing reflex. The three-class classification and data augmentation strategy were crucial for this.
  • Clinical Utility Beyond Detection: The framework not only detects the reflex but also provides precise temporal measurements. This is vital because the timing of the reflex is as important as its presence for diagnosing dysphagia and guiding rehabilitation.
  • Democratizing Expertise: By automating this measurement, the framework has the potential to significantly improve diagnostic consistency, especially for less experienced clinicians, by reducing inter-rater variability.
  • Foundation for Broader Analysis: While this study focuses on response time, the underlying framework could be extended to analyze other critical spatiotemporal parameters in VFSS, such as laryngeal elevation or transit times.

Key Results

The machine learning framework demonstrated high performance:

  • Detection Accuracy: The average success rate for detecting the "during the swallowing reflex" class was 98.2% for the training set and 97.5% for the validation set.
  • Temporal Accuracy: The average difference between the predicted and ground-truth detection times was:
  • Starting point: 0.210 seconds (validation set)

    • End point: 0.056 seconds (validation set) These errors are within the range of normal variation for swallowing reflex times in healthy individuals.
  • Inter-rater Reliability: The study also established strong inter-rater reliability (ICC=0.968) for manual measurements by experts, providing a robust benchmark for their automated system.

Limitations and Future Work

The authors acknowledge several limitations:

  • Sample Size: The study used a relatively small sample size (27 participants, 207 clips). While the results are promising, a larger dataset would further validate the framework's generalization capabilities.
  • Scope of Analysis: The current framework focuses solely on the response time of the pharyngeal swallowing reflex. Future work could expand this to include other spatiotemporal parameters of the entire swallowing process (oral, pharyngeal, and esophageal phases).
  • Integration with Existing Tools: While the framework is presented as a standalone tool, its integration with existing VFSS analysis software could enhance its clinical adoption.

The authors suggest that future research could explore larger datasets and incorporate the analysis of other swallowing parameters to create a more comprehensive VFSS interpretation tool.

Summary updated

Abstract

To evaluate clinical features and determine rehabilitation strategies of dysphagia, it is crucial to measure the exact response time of the pharyngeal swallowing reflex in a videofluoroscopic swallowing study (VFSS). However, measuring the response time of the pharyngeal swallowing reflex is labor-intensive and particularly for inexperienced clinicians, it can be difficult to measure the brief instance of the pharyngeal swallowing reflex by VFSS. To accurately measure the response time of the swallowing reflex, we present a novel framework, able to detect quick events. In this study, we evaluated the usefulness of machine learning analysis of a VFSS video for automatic measurement of the response time of a swallowing reflex in a pharyngeal phase. In total, 207 pharyngeal swallowing event clips, extracted from raw VFSS videos, were annotated at the starting point and end point of the pharyngeal swallowing reflex by expert clinicians as ground-truth. To evaluate the performance and generalization ability of our model, fivefold cross-validation was performed. The average success rates of detection of the class "during the swallowing reflex" for the training and validation datasets were 98.2% and 97.5%, respectively. The average difference between the predicted detection and the ground-truth at the starting point and end point of the swallowing reflex was 0.210 and 0.056 s, respectively. Therefore, the response times during pharyngeal swallowing reflex are automatically detected by our novel framework. This framework can be a clinically useful tool for estimating the absence or delayed response time of the swallowing reflex in patients with dysphagia and improving poor inter-rater reliability of evaluation of response time of pharyngeal swallowing reflex between expert and unskilled clinicians.
BibTeX
@inproceedings{lee2020machine,
  author = {Jong Taek Lee and Eunhee Park and Jong-Moon Hwang and Tae‐Du Jung and Donghwi Park},
  title = {{Machine learning analysis to automatically measure response time of pharyngeal swallowing reflex in videofluoroscopic swallowing study}},
  booktitle = {Scientific},
  year = {2020},
  doi = {10.1038/s41598-020-71713-4},
}
Figure