CP-AGCN: Pytorch-based attention informed graph convolutional network for identifying infants at risk of cerebral palsy

Haozheng Zhang, Edmond S. L. Ho, Hubert P. H. Shum

TL;DR

Researchers have developed CP-AGCN, a deep learning system that identifies infants at risk of Cerebral Palsy (CP) using standard RGB video. By combining Graph Convolutional Networks (GCNs) with a novel frequency-binning module and attention mechanisms, the system achieves state-of-the-art accuracy while remaining efficient enough to run on a standard laptop CPU.

What Problem Does This Paper Address?

Cerebral Palsy is a permanent movement disorder, and early intervention is the single most effective way to improve long-term outcomes for affected infants. Currently, the "gold standard" for early detection is the General Movement Assessment (GMA).

While effective, GMA has a major bottleneck: it requires highly trained clinical experts to manually observe an infant’s movements. This makes it time-consuming, expensive, and difficult to scale to the general population, especially in resource-limited areas. The authors aim to automate this process using "consumer-grade" technology—standard cameras and accessible AI—to provide a low-cost, non-intrusive screening tool.

Methods

The CP-AGCN pipeline consists of three main stages:

  1. Pose Extraction: The system uses OpenPose to extract 2D skeletal joint locations from standard RGB video. This turns a raw video file into a structured "graph" of human movement.
  2. Frequency Binning Module: This is a key innovation. Instead of just looking at raw coordinates, the system applies a Fast Fourier Transform (FFT) to analyze movement in the frequency domain. They designed a custom binning method to filter out high-frequency noise (like camera jitter) and focus on the low-to-mid band frequencies where meaningful CP-related movement patterns occur.
  3. Attention-Informed GCN: The core architecture is a Graph Convolutional Network based on the ST-GCN backbone. The authors added an attention mechanism that learns to weight different joints and frequency features. This not only improves accuracy but also provides interpretability, allowing clinicians to see which body parts or movement frequencies the AI is prioritizing in its decision.

Datasets

The model was validated on two key datasets:

  • MINI-RGBD: A synthetic dataset of infant movements with ground-truth annotations.
  • RVI-38: A real-world dataset specifically designed for infant movement analysis.

The authors noted that their system achieved state-of-the-art performance on both, proving its robustness across both simulated and real-world data.

Key Insights and Ideas

  • Frequency over Space: The paper argues that analyzing movement frequencies is more effective than analyzing raw spatial coordinates. By "binning" these frequencies, the model can distinguish between normal fidgety movements and the specific tremors or stiffness associated with CP.
  • Interpretability is a Requirement, Not a Feature: In medical AI, "black box" models are often rejected. By using attention weights, the researchers ensure that the model can explain why it flagged an infant as high-risk (e.g., highlighting specific abnormal leg movements).
  • Efficiency for the Clinic: One of the most impressive aspects is the computational efficiency. While training requires a GPU (like an RTX 3080), the inference (running the prediction) takes only 45 seconds for a 33-second video on a standard Intel i7 CPU. This makes it viable for use in a typical hospital environment without specialized hardware.

Key Results

  • Performance: The system achieved state-of-the-art (SOTA) results on both the MINI-RGBD and RVI-38 datasets.
  • Speed: Training takes roughly 50 minutes for a standard dataset. At run-time, the system supports "interactive-time" prediction, meaning a clinician could record a video and get a result in less than a minute.
  • Low Cost: Because it only requires RGB video (no depth sensors or wearable markers), the system is significantly more accessible than previous automated solutions.

Limitations and Future Work

Despite the strong results, the authors identify several areas for improvement:

  • Pose Estimation Dependency: The system’s accuracy is heavily reliant on the quality of the initial pose extraction (OpenPose). If the infant is obscured or the lighting is poor, the "skeleton" data may be inaccurate.
  • Hardware Generalization: Future versions aim to improve compatibility with "dirty" video data, such as footage from unfocused or blurry cameras, by implementing a hardware abstraction layer.
  • Embedded Integration: The researchers are looking into moving the model onto embedded systems (like pixel processor arrays) to allow for real-time, autonomous disease monitoring without a PC.
  • Broader Application: The authors suggest this frequency-based GCN approach could be extended to other motor disorders, such as Parkinson’s disease, where tremors and gait patterns are primary diagnostic markers.

Summary updated

Abstract

Early prediction is clinically considered one of the essential parts of cerebral palsy (CP) treatment. We propose to implement a low-cost and interpretable classification system for supporting CP prediction based on General Movement Assessment (GMA). We design a Pytorch-based attention-informed graph convolutional network to early identify infants at risk of CP from skeletal data extracted from RGB videos. We also design a frequency-binning module for learning the CP movements in the frequency domain while filtering noise. Our system only requires consumer-grade RGB videos for training to support interactive-time CP prediction by providing an interpretable CP classification result.
BibTeX
@article{zhang2022cp,
  author = {Haozheng Zhang and Edmond S. L. Ho and Hubert P. H. Shum},
  title = {{CP-AGCN: Pytorch-based attention informed graph convolutional network for identifying infants at risk of cerebral palsy}},
  journal = {Softw. Impacts},
  year = {2022},
  doi = {10.1016/j.simpa.2022.100419},
}

Loading related content...

Figure