Abstract
Parkinson's Disease (PD) is a common neurodegenerative disease which impacts millions of people around the world. In clinical treatments, freezing of gait (FoG) is used as the typical symptom to assess PD patients' condition. Currently, the assessment of FoG is usually performed through live observation or video analysis by doctors. Considering the aging societies, such a manual inspection based approach may cause serious burdens on the healthcare systems. In this study, we propose a pure video-based method to automatically detect the shuffling step, which is the most indistinguishable type of FoG. Firstly, the RGB silhouettes which only contain legs and feet are fed into the feature extraction module to obtain multi-level features. 3D convolutions are used to aggregate both temporal and spatial information. Then the multi-level features are aggregated by the feature fusion. Skip connections are implemented to reserve information of high resolution and period-wise horizontal pyramid pooling is utilized to fuse both global context and local features. To validate the efficacy of our method, a dataset containing 268 normal gait samples and 362 shuffling step samples is built, on which our method achieves an average detection accuracy of 90.8%. Besides shuffling step detection, we demonstrate that our method can also assess the severity of walking abnormity. Our proposal facilitates a more frequent assessment of FoG with less manpower and lower cost, leading to more accurate monitoring of the patients' condition.
Introduction
Freezing of Gait (FoG) is a debilitating symptom of Parkinson’s Disease (PD) where patients feel as if their feet are "glued to the floor." A specific, mild form of this is the shuffling step, where the normal heel-to-toe walking pattern is disrupted—often the toe hits the ground first.
Currently, assessing these symptoms requires specialized doctors to perform manual inspections, either in person or by reviewing video. This is labor-intensive, expensive, and usually only happens a few times a year. While wearable sensors can track gait, they are often cumbersome for patients to use at home. This research proposes an automated, "vision-only" system that works with standard video (like from a smartphone) to make frequent monitoring accessible and objective.
Methodology
The proposed system utilizes a two-stage deep learning pipeline designed to capture the subtle temporal nuances of gait abnormalities.
Feature Extraction
The system processes "RGB silhouettes" (masked images) of a patient's legs and feet. It uses 3D Convolutions (specifically C3D-cells) to process sequences of frames. Unlike standard 2D AI, 3D convolutions capture the temporal relationship between frames—crucial for shuffling, where the specific timing of toe vs. heel strikes is the defining feature. The researchers found that cropping the video to show only the lower quarter of the body yielded the best results, removing "noise" from upper body movements and protecting patient privacy.
Feature Fusion
To handle the subtle differences between a normal gait and a shuffle, the model employs two key architectural strategies:
- UP Block: Uses skip connections to preserve high-resolution details from earlier layers, ensuring the fine movements of the toes aren't lost.
- Period-wise Horizontal Pyramid Pooling (PHPP): This novel pooling method analyzes the gait at different time scales and spatial segments, allowing the model to "see" both the quick moment a foot hits the floor and the overall rhythm of the walking cycle.
Experimental Setup and Dataset
The primary dataset consisted of 147 "Timed Up-and-Go" (TUG) videos collected from 18 PD patients and 42 healthy controls. This yielded 362 shuffling samples and 268 normal gait samples. To balance the dataset and improve robustness, the researchers added 96 negative (normal) gait samples from the public CASIA gait dataset. All samples were scored by professional clinicians using the UPDRS (Unified Parkinson's Disease Rating Scale) standard.
Key Results and Insights
- Detection Accuracy: The proposed method achieved 90.8% accuracy in detecting shuffling steps, significantly outperforming existing models like GaitSet (84.9%) and JGR-GCNN (79.1%).
- Severity Assessment: Beyond binary detection, the model achieved 84.2% accuracy in classifying the gait into three levels of severity based on clinical standards.
- Temporal Importance: The study confirms that temporal aggregation is mandatory for PD diagnosis because the primary cue for shuffling is the inverted hitting order of the foot.
- Efficiency: The model is fast enough for real-time applications, processing a 25-frame sequence in only a few milliseconds on a standard high-end GPU (RTX 2080Ti).
Conclusion
While the current dataset is relatively small (18 patients), the results demonstrate a high-performance, low-cost path toward remote, continuous patient monitoring. Future work will focus on expanding the dataset to improve generalization. The ultimate goal is to integrate this technology into mobile platforms, allowing patients to perform a simple TUG test at home and send automated reports to their doctors for more precise, data-driven treatment adjustments.
Summary updated