Interactive Video Acquisition and Learning System for Motor Assessment of Parkinson's Disease

Yunyue Wei, Bingquan Zhu, Chen Hou, Chen Zhang, Yanan SuiIJCAI 2021 · OpenReview

Abstract

Diagnosis and treatment for Parkinson's disease rely on the evaluation of motor functions, which is expensive and time consuming when performing at clinics. It is also difficult for patients to record correct movements at home without the guidance from experienced physicians. To help patients with Parkinson’s disease get better evaluation from in-home recorded movement videos, we developed an interactive video acquisition and learning system for clinical motor assessments. The system provides real-time guidance with multi-level body keypoint tracking and analysis to patients, which guarantees correct understanding and performing of clinical tasks. We tested its effectiveness on healthy subjects, and the efficiency and usability on patient groups. Experiments showed that our system enabled high quality video recordings following clinical standards, benefiting both patients and physicians. Our system provides a novel learning-based telemedicine approach for the care of patients with Parkinson’s disease.

Introduction

The gold standard for assessing Parkinson’s Disease (PD) is the MDS-UPDRS (Unified Parkinson’s Disease Rating Scale) Part III, which requires patients to perform specific motor tasks such as finger tapping, hand movements, or walking. Traditionally, these assessments occur in a clinical setting, imposing a significant time and cost burden on both patients and healthcare providers.

While telemedicine is a promising alternative, "home-made" videos are frequently unusable for clinical diagnosis due to poor lighting, improper framing, or patient misunderstanding of instructions. Furthermore, the lack of standardized, high-quality video datasets hinders the development of automated diagnostic tools. PD-GUIDER, developed by researchers at Tsinghua University, aims to bridge this gap by acting as a "virtual physical therapist" that guides the recording process through real-time AI feedback.

System Architecture and Workflow

PD-GUIDER is an Android-based application that utilizes a "Sense-Guide-Record" workflow to ensure data quality. The system's core is an on-device AI engine running TensorFlow Lite models, specifically PoseNet for full-body tracking and HandPose for fine-grained hand movement analysis. This allows for immediate processing without the latency of cloud-based analysis.

The PD-GUIDER system workflow, illustrating the path from task selection to server upload.

The application guides users through a structured process: selecting a movement, viewing instructional videos, and entering a calibration phase before recording begins. By automating the "start" and "stop" triggers based on physical positioning, the app removes the technical friction and cognitive load often associated with operating a camera for elderly patients.

Real-time Calibration and Guidance

To solve the problem of clinically invalid recordings, the system employs two layers of calibration:

  • Soft Calibration: A preset silhouette on the screen helps the user position themselves correctly before starting.
  • Hard Calibration: The system automatically calculates environmental luminance and checks if the AI-detected body bounding box matches preset "movement boundary boxes."
The calibration interface providing immediate feedback on lighting conditions and user positioning.

If the user is out of frame or the room is too dark, the system provides real-time audio-visual feedback (e.g., "Dim light, please adjust") and prevents recording until the conditions are met. This task-specific logic ensures that for gait assessments, the user starts far enough away to capture a full walking cycle, while for posture tasks, the entire body remains visible.

Experimental Results

The system's effectiveness was evaluated in a study involving 5 healthy subjects and 21 patients with PD. The patients were divided into a guided group (using PD-GUIDER) and a baseline group (recording without guidance). The results demonstrated significant improvements across all metrics:

  • Accuracy: The movement error rate (performing the wrong action) dropped from 16.7% to 9.3%.
  • Video Quality: The proportion of videos missing critical body parts plummeted from over 50% to below 2%.
  • Efficiency: The average time required for a full recording session was reduced from 57 minutes to 34 minutes.
  • User Satisfaction: Patients rated the app highly, finding the interactive guidance and automated triggers particularly useful, even among those with cognitive challenges.

Conclusion and Future Work

PD-GUIDER demonstrates that real-time, on-device AI can significantly improve the quality and efficiency of remote motor assessments for Parkinson's Disease. By closing the feedback loop during recording, the system prevents the collection of "garbage data" and facilitates more reliable telemedicine.

Future iterations of the system aim to include multi-person tracking for patients requiring caregiver assistance, as well as the integration of speech and drawing-based assessments. The ultimate goal is to transition from high-quality data acquisition to fully automated symptom rating using the captured keypoint data.

Summary updated

Abstract

Diagnosis and treatment for Parkinson's disease rely on the evaluation of motor functions, which is expensive and time consuming when performing at clinics. It is also difficult for patients to record correct movements at home without the guidance from experienced physicians. To help patients with Parkinson’s disease get better evaluation from in-home recorded movement videos, we developed an interactive video acquisition and learning system for clinical motor assessments. The system provides real-time guidance with multi-level body keypoint tracking and analysis to patients, which guarantees correct understanding and performing of clinical tasks. We tested its effectiveness on healthy subjects, and the efficiency and usability on patient groups. Experiments showed that our system enabled high quality video recordings following clinical standards, benefiting both patients and physicians. Our system provides a novel learning-based telemedicine approach for the care of patients with Parkinson’s disease.
BibTeX
@inproceedings{wei2021interactive,
  author = {Yunyue Wei and Bingquan Zhu and Chen Hou and Chen Zhang and Yanan Sui},
  title = {{Interactive Video Acquisition and Learning System for Motor Assessment of Parkinson's Disease}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=srImWvnQc6},
}
Figure