FINE TUNING OF A PRETRAINED GAIT MODEL FOR FRAILTY CLASSIFICATION PREDICTION

Lisa K. McDaniel, Ime Essien, Yuxiang Guo, Ayush Gupta, Vineet Shenoy, Peter Abadir, Rama ChellappaInnovation 2024 · DOI

TL;DR

This paper tackles the challenge of early frailty detection by fine-tuning a pre-trained gait analysis model (OpenGait) on a novel medical dataset. By combining silhouette and RGB video data, they aim to provide clinicians with a more efficient and accessible tool for assessing frailty, a critical indicator of health decline in older adults.

What Problem Does This Paper Address?

Frailty is a significant health concern, characterized by decreased physiological reserve and increased vulnerability to adverse outcomes like falls, hospitalization, and mortality. Early detection is crucial for timely intervention and management. While deep learning has shown promise in medical applications, its use in gait-based frailty detection is hampered by the scarcity of relevant, labeled medical data. Existing gait analysis models are often trained on surveillance or recognition tasks, not specifically for clinical frailty assessment. This paper aims to bridge this gap by adapting a powerful pre-trained gait model for the specific task of classifying frailty status.

Methods

The core of this work involves fine-tuning a pre-trained gait model, OpenGait, which was originally developed for gait surveillance and recognition. The authors leverage the GaitBase framework, a component of OpenGait. To adapt this model for frailty classification, they employ a multimodal approach, integrating information from both:

  • RGB video frames: Standard visual data capturing the walking motion.
  • Silhouette predictions: Generated using Detectron2, a popular object detection and segmentation framework. Detectron2 is used for panoptic image segmentation and detection, providing outlines (silhouettes) and bounding boxes of individuals in the video.

By combining these two data modalities, the model aims to achieve greater robustness and generalization across different frailty classes. To mitigate the challenges posed by a small dataset, they utilize standard machine learning techniques:

  • Data Augmentation: Artificially increasing the size and diversity of the training data.
  • Hyperparameter Tuning: Optimizing the model's learning parameters.
  • Cross-Validation: A technique to ensure the model's performance is reliable and not overfitted to the training data.

Datasets

The study utilizes a newly collected dataset for frailty classification.

  • Dataset: Data was collected using the marker-less Qualisys Motion Capture system.
  • Participants: Consists of 64 individuals.
  • Classification: Individuals are classified into three frailty categories: non-frail, frail, and pre-frail, based on the Fried Frailty Phenotype criteria.

This dataset is specifically curated for medical frailty assessment, differentiating it from datasets used for general gait recognition.

Key Insights and Ideas

  • Leveraging Pre-trained Models for Clinical Tasks: The paper demonstrates the feasibility of adapting powerful, general-purpose gait analysis models (like OpenGait) for specialized clinical applications like frailty detection, even with limited medical data. This is a significant step towards democratizing advanced AI in healthcare.
  • Multimodal Gait Analysis for Frailty: Combining silhouette information with raw RGB video data appears to be a promising strategy for improving the accuracy and reliability of frailty classification. Silhouettes can capture key kinematic features while RGB data provides richer contextual information.
  • Addressing Data Scarcity: The successful application of data augmentation, hyperparameter tuning, and cross-validation highlights effective strategies for building robust models in data-limited medical domains.

Key Results

The paper reports that their fine-tuned multimodal gait model enhances robustness and generalization across the three frailty classes. While specific quantitative accuracy metrics are not detailed in the provided abstract, the authors state that their approach aims to improve classification accuracy and offers a time-efficient method for clinicians.

Limitations and Future Work

  • Dataset Size: The dataset consists of only 64 individuals, which is a significant limitation for deep learning models. While data augmentation and other techniques were employed, a larger and more diverse dataset would likely lead to even better performance and generalizability.
  • Future Work: The authors explicitly mention that future steps include incorporating metadata such as height and weight to further enhance prediction accuracy. This suggests that integrating demographic and anthropometric information alongside gait analysis could lead to more comprehensive frailty assessments.

Summary updated

Abstract

Abstract This study presents a multimodal gait model designed to identify the frailty status of individuals based on their gait observed in video footage. Frailty is defined as a clinically recognizable state of increased vulnerability resulting from age-related declines in reserve and function across multiple interrelated physiological systems. These risks manifest as an increased likelihood of falls, hospitalization, need for long-term care, disability, and mortality. Although deep learning has advanced in medical applications, its potential in early frailty detection through gait analysis is limited by the need for extensive data, which is currently limited. To address this issue, we leveraged the OpenGait framework, GaitBase, a pretrained gait model typically used for gait surveillance and recognition applications, by fine-tuning it with medical data for frailty classification. Data were collected using the marker-less Qualisys Motion Capture system, resulting in a dataset of 64 individuals classified by frailty status—non-frail, frail, and pre-frail—according to the Fried Frailty Phenotype. Detectron2 models were used for panoptic image segmentation and detection, generating silhouette and box predictions from video frames. By integrating both silhouette and RGB data, our model enhances robustness and generalization across the three frailty classes. To address the challenges of a small dataset, we employed data augmentation, hyperparameter tuning, and cross-validation to improve classification accuracy. This model aims to advance frailty classification using deep learning, offering a time-efficient method for clinicians to assess frailty. Future steps include incorporating metadata such as height and weight to further enhance prediction accuracy.
BibTeX
@inproceedings{mcdaniel2024fine,
  author = {Lisa K. McDaniel and Ime Essien and Yuxiang Guo and Ayush Gupta and Vineet Shenoy and Peter Abadir and Rama Chellappa},
  title = {{FINE TUNING OF A PRETRAINED GAIT MODEL FOR FRAILTY CLASSIFICATION PREDICTION}},
  booktitle = {Innovation},
  year = {2024},
  doi = {10.1093/geroni/igae098.3984},
}
Figure