When a stroke survivor relearns to reach, or a person with Parkinson's disease slowly loses fluid gait, the clinically meaningful change is often not in any single joint. It is in how joints move together. The trunk starts to substitute for the shoulder. The hip and knee lose their usual timing. One side of the body stops mirroring the other. Current clinical machine learning handles this badly in two opposite ways. Interpretable movement classifiers score each joint separately and look at one session at a time. Longitudinal progression models track brain scans or a single aggregate clinical score, and never look at movement structure at all. We propose to make the coupling between joints the unit that is measured at every visit, tracked over months, and shown to the clinician. Concretely, the coupling is a small set of named, shrinkage-stabilized correlations and synergy counts estimated from repeated trials within each visit.
The design is shaped by the fact that clinical motion cohorts are tiny, with tens of patients and two to five visits each. So the method is deliberately low-capacity. Each visit contributes many repeated movements, which is enough to estimate a covariance across joints if it is pulled toward a population prior. Patients are then placed on a shared progression timeline with a patient-specific delay and speed. The explanation a therapist receives is a sentence like "trunk–shoulder coupling rose from 0.2 to 0.6 between visits 2 and 3 while elbow–shoulder coupling fell." No unexplained latent state stands behind it.
Core Claim
Change in inter-joint coupling structure, estimated per visit with shrinkage and aligned across patients in disease time, predicts future clinical change better than three alternatives at the same small sample sizes:
- change in per-joint features,
- the clinical score's own history,
- end-to-end sequence models over raw skeletons.
It also names the specific coupling responsible in a way clinicians can verify. If per-joint feature deltas do as well, coordination is not the right longitudinal unit, and we will say so. The experiments are built to make that outcome visible.
Relation to Prior Work
Closest work. The nearest work is Online detection of compensatory strategies in human movement with supervised classification: a pilot study.
- What it does: computes jerk, power, effort and torque rate for 10 joints from motion capture, then shows that logistic regression on these 40 features beats LSTMs on 600 acted trajectories from 6 physiotherapists. It traces each decision to individual joint weights.
- What it leaves open: it explicitly treats joints independently and is cross-sectional.
- How we differ: we keep its philosophy (physically meaningful features, linear readout, few subjects) but replace per-joint quantities with between-joint quantities, and add the time axis.
Evidence that coupling matters. Two papers motivate the focus on coupling:
- A Neural Network-Based Approach to Robot Motion Control shows that controlling robot leg joints independently accumulates error that a joint model removes.
- Probabilistic Movement Primitives represents a movement as a Gaussian over basis-function weights with full covariance across joints. That covariance is what lets remaining joints compensate when one is perturbed.
We reuse the movement-primitive representation as a measurement device rather than a controller.
A constraint from the generative-model literature. Full-covariance and diffusion-based joint action models such as Imitating Human Behaviour with Diffusion Models capture rich correlations but need dense demonstrations. We therefore estimate only a low-dimensional, named summary of the covariance rather than a generative model.
Methods for the longitudinal half. - Conditional Neural ODE for Longitudinal Parkinson's Disease Progression Forecasting handles irregular PPMI visit gaps (0.61–2.27 years) by learning a patient-specific time shift and progression speed over cortical-thickness features. We borrow the shift-and-speed alignment. We replace the neural ODE with a monotone population curve, because our cohorts are smaller and our coordinates are already interpretable. - DeltaNet and the image difference graphs of Hu et al. show that explicitly subtracting the prior visit avoids attending to unchanged anatomy. We apply the same delta principle to coupling matrices.
Clinical motion and progression work that stays black-box or cross-sectional. - AdaMedGraph forecasts Parkinson's progression from clinical scores via patient-similarity graphs. It treats motor function as a single scalar. - Vision-based Estimation of MDS-UPDRS Gait Scores maps pose sequences to gait scores with a convolutional network over joint distances. It is accurate on 30 patients but opaque and single-visit.
Interpretability by structured bottleneck. The interpretability stance follows two papers that force predictions through clinically named intermediates:
- PN-SAMP routes malignancy prediction through nodule attributes.
- Pisov et al. predicts an explicit midline curve with confidence intervals.
Our intermediates require no extra annotation, because they are computed from kinematics, not labeled by experts. This sidesteps the annotation cost those bottleneck models incur.
Research Program
Per-visit measurement. For patient $i$ at visit $v$ we record $K$ repetitions of a standardized task, such as reaching or a walking bout. Each repetition is time-normalized and projected onto movement-primitive basis functions per joint angle, giving a weight vector per trial. From the $K$ vectors we form the sample covariance $S_{iv}$ and shrink it toward a population covariance $\Sigma_0$ estimated from healthy reference data:
$$\hat\Sigma_{iv} = (1-\lambda_{iv})\,S_{iv} + \lambda_{iv}\,\Sigma_0,$$
where $\lambda_{iv}\in[0,1]$ is set analytically by the Ledoit–Wolf rule from $K$ and the dimension. This is what makes estimation stable when $K$ is 10–20.
Coordination concepts. From $\hat\Sigma_{iv}$ we compute a fixed vocabulary, $c_{iv}$, of a dozen or so quantities:
- partial correlations for clinically meaningful joint pairs (trunk–shoulder, shoulder–elbow, hip–knee, knee–ankle, left–right homologues);
- their signs, since the negative couplings that preserve balance are exactly what per-joint summaries miss;
- the participation ratio of the eigenvalues, an "effective number of synergies."
Each concept carries a bootstrap interval over trials, in the spirit of Pisov et al.'s confidence bands.
Longitudinal alignment. Each concept follows a shared monotone spline $\mu_k$ in aligned disease time:
$$c_{ivk} = \mu_k\big(\gamma_i (t_{iv} - \tau_i)\big) + \epsilon_{ivk},$$
where $t_{iv}$ is calendar time since enrollment, $\tau_i$ a patient time shift, $\gamma_i$ a patient progression speed, and $\epsilon$ residual noise. This is fit as a nonlinear mixed-effects model with priors on $\gamma_i$ and $\tau_i$.
Prediction and explanation. The outcome head is sparse logistic or ordinal regression on two inputs: the concept deltas between visits, and the patient's inferred $\gamma_i$. It predicts next-visit clinical change. The explanation is the set of concepts with non-zero weight and large observed delta, each with its interval.
Baselines
Naive reference. The do-nothing reference predicts that the next clinical score equals the current one, or the majority change class. Any method that does not beat persistence on a cohort with slow progression is not useful.
Strongest interpretable small-sample competitor. This is Das et al.'s pipeline: the 40 per-joint energy features with logistic regression, extended longitudinally by feeding visit-to-visit deltas of those same features into the identical sparse head. On its own acted dataset it reports a misclassification rate "<0.14" and mean Brier score "<0.12" for logistic regression (Das et al.). We run it on identical trials and splits (leave-one-subject-out), with the same regularization search budget. Beating it means lower Brier score and misclassification on longitudinal change prediction, with a paired bootstrap interval excluding zero.
Strongest deep competitor. This is an OF-DDNet-style temporal convolutional model on joint distances. It reports macro-F1 0.83, AUC 0.90 and 81% balanced accuracy on 30 Parkinson's patients for cross-sectional gait scoring (Vision-based Estimation of MDS-UPDRS Gait Scores). For the longitudinal setting we add a GRU over visit embeddings. It receives the same pose inputs, the same hyperparameter budget measured in GPU-hours, and early stopping on inner folds. On the cross-sectional task, matching it within its reported F1 while staying fully interpretable would count as a win. On the longitudinal task, beating it outright would.
Score-only competitor. Clinical scores dominate in AdaMedGraph: removing clinical data dropped 12-month Hoehn & Yahr AUC from 0.774 to 0.448, and 12-month MDS-UPDRS III AUC is 0.714 on PPMI (AdaMedGraph). So we run gradient-boosted trees on prior clinical scores and demographics alone. We also run that model with our concepts added, which tests whether kinematics contributes anything beyond the scale.
Ablations isolating the key ingredients. - Unshrunk covariance ($\lambda=0$). - Raw pairwise Pearson correlations of joint angles without the primitive projection. - Concepts used cross-sectionally without the time alignment ($\gamma_i=1,\tau_i=0$).
For the longitudinal tasks, there is no reported result on Lacuna for any coordination-based progression model, nor for CNODE in numeric form; its gains are reported only as plots.
Why not just…?
Why not just add pairwise joint correlations to Das et al.'s logistic regression? This is the most serious threat. It is close to what we propose cross-sectionally, and we treat it as an ablation rather than dismissing it. Our claim does not rest on correlations being exotic features. It rests on three things together:
- shrinkage makes them stable at realistic trial counts;
- deltas of coupling, aligned in disease time, carry the longitudinal signal;
- the outputs are named and bounded.
If raw correlations with no shrinkage and no alignment match the full method, the contribution shrinks to "use coupling features." The sample-size experiment below is designed to show whether unshrunk estimates become noise at $K\le 15$.
Why not just take deltas of per-joint features across visits? It is the simplest longitudinal extension of the strongest existing interpretable method, and it may work. Das et al. concede that per-joint features miss coordination, and Grasemann et al. show that independent joint treatment misses cross-talk. Still, whether this matters for clinical prediction is untested. The head-to-head on the longitudinal cohort settles it. We will also report cases where the two disagree and which one therapists find correct.
Why not just forecast the clinical score with boosting or AdaMedGraph? Scores are strong predictors, but they are coarse and rater-dependent; the gait-scoring paper notes intrinsic ambiguity in intermediate classes. They also say nothing about which compensation to target in therapy. If concepts add no predictive value on top of scores, the method's value is purely explanatory. We would report that explicitly rather than claim a forecasting gain.
Why not just train an end-to-end sequence model on skeletons across visits? With 2–5 visits per patient and tens of patients, such models face the overfitting documented by Dong et al., where 45 volumes defeated unconstrained networks. CNODE's comparison also found that GRUs and Transformers underperform on irregular PPMI visits. They remain a fair competitor and are run at matched budget.
Proposed Experiments
Experiment 1: cross-sectional sanity check on the acted dataset. We use Das et al.'s 600 acted trajectories with leave-one-subject-out over the 6 physiotherapists. The question is whether coordination concepts plus energy features reduce misclassification and Brier score below the reported logistic-regression figures. A second check is whether attribution names trunk–shoulder coupling for torso-leaning trials and shoulder-elevation coupling for shrugging trials. This tests whether the concepts mean what we say. Trial-count subsampling ($K$ from 3 to 20 per condition) measures concept stability, as bootstrap interval width and test–retest agreement, with and without shrinkage.
Experiment 2: ground-truth recovery on semi-synthetic data. Starting from the healthy-mode trajectories, we inject gradual, known changes in specific couplings across simulated visits, with irregular gaps drawn from the PPMI range of 0.61–2.27 years. The method must recover which coupling drifted, plus each simulated patient's speed and shift. Per-joint deltas and the GRU must do the same. This is the only setting with true answers for the explanation.
Experiment 3: cross-sectional Parkinson's gait. On the 30-patient gait video cohort, we use the same monocular 3D pose pipeline and predict MDS-UPDRS gait sub-scores from concepts with an ordinal sparse head. We compare against the reported 0.83 macro-F1. A small loss in F1 alongside full per-concept explanations would be an acceptable trade; a large loss would mean coupling misses most gait-severity information.
Experiment 4: the decisive longitudinal test. No dataset in the file has repeated kinematics over months. We therefore propose collecting one: a cohort of roughly 25–30 subacute stroke patients recorded with consumer video at four visits over six months. Each visit includes 15 repetitions of reaching and gait tasks, alongside the routine clinical motor scale.
The primary endpoint is leave-one-patient-out prediction of next-visit clinical change (improved, unchanged, worse), scored by Brier score and AUC against all baselines. The decisive comparison is coupling deltas versus per-joint feature deltas at matched head complexity. If coupling does not win, the paper reports that per-joint tracking suffices.
The secondary endpoint is blinded therapist review. For each patient, therapists see the top concept changes from our method and from per-joint deltas, and rate which matches their observed compensation. This tests whether the explanations are clinically right, not merely sparse.
References
- Online detection of compensatory strategies in human movement with supervised classification: a pilot study
- A Neural Network-Based Approach to Robot Motion Control
- Probabilistic Movement Primitives
- Imitating Human Behaviour with Diffusion Models
- Conditional Neural ODE for Longitudinal Parkinson's Disease Progression Forecasting
- DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis
- Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering
- Personalized progression modelling and prediction in Parkinson’s disease with a novel multi-modal graph approach
- Vision-Based Estimation of MDS-UPDRS Gait Scores for Assessing Parkinson's Disease Motor Severity
- Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction
- Incorporating Task-Specific Structural Knowledge into CNNs for Brain Midline Shift Detection
- A Left Ventricular Segmentation Method on 3D Echocardiography Using Deep Learning and Snake