Markerless Motion Analysis for Automated Clinical Diagnosis

This direction is merged into another direction: manual clinical assessment bottleneck. Its last version stays readable and citable here.

Version 1 of 118 papers18 conceptsfrozen 2026-10-04

In clinical neurology and sports medicine, movement is a vital sign. The way an infant kicks or an athlete lands a jump can reveal early indicators of cerebral palsy or the risk of a ligament tear. However, the "gold standard" for assessing these movements has long been manual observation by highly trained experts. This creates a massive bottleneck: expert time is expensive, subjective, and difficult to scale to the general population.

A growing body of research is replacing this manual "eye test" with automated pipelines that use standard video to extract, quantify, and interpret human motion. By combining markerless pose estimation with domain-specific statistical models, researchers are turning casual video into a precise diagnostic tool.

The Expert Bottleneck and the Need for Objectivity

Traditional clinical assessments, such as the General Movement Assessment (GMA) for infants, rely on clinicians watching hours of video to identify subtle patterns like "fidgety movements." As noted by Zhiyuan Luo and Ali Gooya in UDF-GMA: Uncertainty Disentanglement and Fusion for General Movement Assessment, this process requires constant recalibration to maintain accuracy and is inherently limited by human fatigue and subjectivity.

The goal of markerless motion analysis is to digitize this expertise. Instead of a clinician noting a "poor repertoire" of movement, a computer vision system can quantify the exact velocity, acceleration, and symmetry of limbs. This transition from qualitative description to quantitative data allows for the detection of developmental benchmarks, such as the "inverted U-shaped" patterns in movement frequency identified by Naoki Kinoshita and colleagues in Longitudinal assessment of U-shaped and inverted U-shaped developmental changes.

From Pixels to Anatomical Models

The first challenge in automated assessment is accurately "seeing" the body. While general-purpose tools like OpenPose work well for adults, they often fail on infants due to different body proportions and frequent self-occlusion (e.g., a baby tucking their legs).

To solve this, Nikolas Hesse, Michael J. Black, and their team developed SMIL (Skinned Multi-Infant Linear model). As detailed in Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences, SMIL is a statistical 3D body model learned from thousands of infant scans. By fitting this model to noisy RGB-D data, the system can "infer" the position of a limb even if it is partially hidden, providing a robust 3D skeleton for analysis.

Similarly, Haomiao Ni and Yuan Xue introduced SiamParseNet, a framework that uses semi-supervised learning to track infant body parts even when labeled data is scarce. By using synthetic data generators like FVGAN, they bridge the gap between "perfect" lab data and the messy reality of clinical video recordings.

Quantifying Pathology through Temporal Patterns

Once the skeleton is extracted, the next step is classification. Movement disorders are rarely defined by a single frame; they are defined by how movement evolves over time.

Researchers use various architectures to capture these temporal signatures:

  • Two-Stream Networks: Manli Zhu and Hubert P. H. Shum developed a 2s-CNN that analyzes both individual joint positions and the relative distances between joints over time, achieving high accuracy in gait-based diagnosis.
  • Time-Series Ensembles: In A Novel Video-Based Methodology for Automated Classification of Dystonia and Choreoathetosis, Helga Haberfehlner and colleagues combined DeepLabCut with HIVE-COTE 2.0, an ensemble learner designed specifically for complex time-series data, to distinguish between different types of cerebral palsy.
  • Deviation Analysis: For physical therapy, Tal Hakim and Ilan Shimshoni created A-MAL, which learns what a "correct" movement looks like and then calculates the mathematical deviation of a patient's attempt to provide a recovery score.

Building Clinical Trust through Explainability

For these systems to be adopted in hospitals, they cannot be "black boxes." Clinicians need to know why a model flagged a movement as abnormal.

Kevin D. McCay and his team addressed this in Towards Explainable Abnormal Infant Movements Identification by creating a visualization framework. Instead of just outputting a risk percentage, their system highlights specific body parts—such as an asymmetrical arm movement—that contributed to the prediction. This allows the AI to act as a "second opinion" that directs the clinician's attention to specific diagnostic markers.

Furthermore, addressing data noise is critical. The UDF-GMA framework by Luo and Gooya explicitly disentangles "aleatoric" uncertainty (noise in the video, like poor lighting) from "epistemic" uncertainty (the model's own lack of knowledge). By quantifying how sure the model is about its own prediction, it provides a safety mechanism for clinical decision-making.

Expanding the Scope of Markerless Screening

While much of the focus is on complex neurological disorders, the same principles are being applied to structural and orthopedic risks. Michael J. Wan and Sarah Ostadabbas demonstrated that deep learning can screen for torticollis (neck twisting) by measuring facial and upper-body symmetry from simple photographs. In sports medicine, Ziyu Gong and colleagues used OpenPose to automate the Landing Error Evaluation System (LESS), turning a manual 2D video inspection into an objective score for ACL injury risk.

By moving clinical assessment from the specialized lab to standard video, this research direction promises a future where developmental and physical screening is continuous, objective, and accessible to all.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_markerless_motion_analysis_for_automated_9d41823d,
  title = {Markerless Motion Analysis for Automated Clinical Diagnosis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/markerless-motion-analysis-for-automated-clinical-diagnosis/rdr_9d41823d059445ed8765093ed824e236},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_markerless_motion_analysis_for_automated_9d41823d_v1,
  title = {Markerless Motion Analysis for Automated Clinical Diagnosis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/markerless-motion-analysis-for-automated-clinical-diagnosis/rdr_9d41823d059445ed8765093ed824e236/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}