Who Was Recorded, or How? Balanced Cross-Site Evaluation of Part-Based Gait Models

Models that score Parkinson's disease severity from 3D walking data lose accuracy when tested at a clinic they were not trained on. The usual explanation is that each clinic captures motion differently: different cameras, marker systems, and pose-reconstruction software. The usual fix is to fine-tune on a little data from the new clinic. There is a second explanation that nobody has separated from the first. Clinics also enroll different patients, with different severity mixes, different shares of patients tested on or off their medication, and different rates of freezing episodes. A model can look worse at a new site simply because that site's patients are harder cases. Separating these two causes matters in practice. If the gap comes from who was recorded, fine-tuning on local recordings is the wrong remedy. If it comes from how they were recorded, the next question is where on the body the recording artifacts sit.

We propose to make this separation with a standard tool from observational statistics, the propensity score. Here it is the estimated probability that a patient belongs to the held-out clinic, given clinical covariates such as severity, medication state, and freezing status. Reweighting the training clinics' test patients by this score lets them stand in for the new clinic's patient mix. The accuracy that remains lost after reweighting is attributed to acquisition. We then localize that remaining shift on the skeleton. A motion encoder split into body-part branches (trunk, lower limbs, upper limbs, head) lets us ask which branch still tells clinics apart once patients are matched. That answer points to a targeted fix: adapt only the flagged branches.

Core Claim

The cross-center drop reported on CARE-PD splits into two terms. The first is a population term, which inverse-propensity reweighting of in-distribution test data can estimate. The second is an acquisition term, which is the shift left over after reweighting. We hypothesize that the acquisition term is concentrated in a few kinematic sub-chains, most plausibly the distal limbs and the root trajectory, where video-based reconstruction and marker-based reconstruction differ most. If so, adapting only those parts' representations should recover the cross-site gap at least as well as whole-model fine-tuning, with less risk of overfitting a small site.

The decomposition can be written in one line. Let $S$ be the training sites and $t$ the held-out site. Let $Z$ be the clinical covariates. Let $\hat e(z)$ be a classifier's estimate of $P(\text{site}=t \mid Z=z)$, fit on pooled patients. Let $R(\cdot)$ be the error of a fixed trained model, evaluated on the indicated data. Then

$$\Delta = \underbrace{R_w(S) - R(S)}_{\text{population}} + \underbrace{R(t) - R_w(S)}_{\text{acquisition}}, \qquad w(z) = \frac{\hat e(z)}{1-\hat e(z)}.$$

Here $R(S)$ is error on held-out patients from the training sites. $R_w(S)$ is that same error with each patient weighted by $w(z)$, so that the training-site patients mimic site $t$'s covariate mix. $R(t)$ is the observed LODO error at the held-out site. The split is only meaningful where the two covariate distributions overlap. We therefore report overlap diagnostics and trim patients whose scores are near 0 or 1, using the trimming rules studied in Weight Trimming and Propensity Score Weighting. We also run a stabilized variant based on Stabilized Inverse Probability Weighting via Isotonic Calibration. For the inference, we will account for the fact that the scores are estimated rather than known, following Matching on the Estimated Propensity Score. All of these papers supply the statistical method. None of them deals with structured motion data.

Relation to Prior Work

The work that motivates the proposal is CARE-PD itself. It pooled 9 cohorts from 8 centers and showed that deep encoders beat handcrafted gait features across centers. It also showed that the gap shrinks with Leave-One-Dataset-Out (LODO) training followed by Multi-dataset In-domain Adaptation (MIDA), its target-site fine-tuning protocol. It never asks whether the gap reflects patient mix. Its own reconstruction result, a drop from 60.7 mm to 7.5 mm per-joint error after fine-tuning on pathological gait, shows that population differences (healthy versus PD) can be enormous.

The closest methodological work is Propensity Score Alignment of Unpaired Multimodal Data. It trains classifiers to predict a known perturbation from each modality, treats the predicted probabilities as a shared coordinate system, and matches samples with optimal transport. We borrow its insight that a treatment classifier yields a useful low-dimensional summary. Our use differs in two ways. First, our propensity score is computed from patient covariates to define a fair evaluation, not to pair samples across modalities. Second, we apply site classifiers per body part to locate where shift lives, which PSA never considers because its data have no skeleton.

The hierarchical side supplies the architecture. Dynamic Compositional Graph Convolutional Network uses separate upper-body, lower-body, and whole-body branches so that arm motion does not corrupt leg dynamics. Take an Emotion Walk pools joint features into limb features and then body features. An Approach to Pose-Based Action Recognition shows that part-based representations confine local corruption. That last property is exactly what makes per-part shift localization possible. These papers decompose the body for accuracy or robustness. We decompose it for attribution.

Research Program

We first build a part-factorized encoder on CARE-PD's SMPL sequences. A MotionAGFormer-style backbone, the encoder CARE-PD uses, is split into trunk, left and right lower limb, left and right upper limb, and head branches, with a fusion head that predicts UPDRS-gait severity on the 0 to 3 scale. A flat version of the same backbone with matched parameter count serves as control.

Second, for each held-out site we fit the covariate propensity model. Its inputs are the UPDRS label distribution, medication state, and freezing status, plus age, sex, and disease duration wherever the site metadata provide them. We compute the population/acquisition decomposition above for both encoders.

Third, we localize the acquisition term. After reweighting patients to balance $Z$, we train a small probe per branch to predict "site $t$ versus training sites" from frozen branch embeddings. The probe's balanced accuracy above chance gives a per-part acquisition fingerprint. Raw-signal checks on per-part joint jitter and root drift accompany it, so the result is not an artifact of the probe.

Fourth, we test the remedy. Part-selective adaptation fine-tunes only the branches flagged in the third step on the same small target set MIDA uses, and leaves the other branches frozen.

Because CARE-PD has no ground truth for this decomposition, we add a controlled testbed on MoVi. MoVi records the same 90 subjects with optical motion capture (processed to SMPL via MoSh++/AMASS) and with synchronized video. That lets us create two synthetic "sites" whose acquisition differs in a known way: mocap-derived SMPL versus video-estimated SMPL from the WHAM pipeline CARE-PD uses. We then independently impose a population shift by sampling subjects on BMI (17 to 32) and sex. The task is 21-way action classification. We can switch each shift on or off, so we know the true contribution of each, and the decomposition and localization can be checked against it before we trust them on clinical data.

Baselines

The naive reference is the raw LODO gap of the flat encoder, with no reweighting and no adaptation. It shows how large the problem looks under the current protocol. The strongest existing method is CARE-PD's MIDA: whole-model target-site fine-tuning of MotionAGFormer. We run it with the official splits, the same target-site sample budget, the same number of gradient steps, and the same hyperparameter search budget as our part-selective adaptation.

On the strength of this baseline: CARE-PD reports MPJPE falling from 60.7 mm to 7.5 mm after fine-tuning on pathological gait. No reported result was found on Lacuna for UPDRS-gait classification F1 or accuracy under LODO or MIDA. We will reproduce those numbers ourselves and report them as the reference.

Beating MIDA means equal or better held-out-site macro-F1 with fewer adapted parameters and lower variance across the 9 held-out folds. The two ablations isolate the key ingredient, which is the localization step. One adapts a randomly chosen set of branches of the same size as the flagged set. The other adapts the flagged branches chosen without covariate balancing. If either matches our method, the propensity-balanced localization is doing no work.

For the decomposition itself, the baseline is a non-causal estimate: regress per-patient error on site indicator plus covariates and read off the site coefficient. Our estimate must recover the known MoVi shift contributions more accurately than this regression does.

Why not just…?

Why not just fine-tune on the new site, as MIDA does? Fine-tuning can shrink the gap without telling us why it existed. When the gap is mostly population shift, fine-tuning on a few local patients risks fitting that site's severity mix rather than learning anything transferable. The decomposition says which situation each fold is in. The part-selective comparison against MIDA at equal budget shows whether knowing the cause pays off. If MIDA wins everywhere, we will report that the attribution has diagnostic value only.

Why not just add the clinical covariates as model inputs, or regress them out? Covariate adjustment changes the model, but it does not tell us how much of the original gap came from patient mix. It also assumes a correct functional form. Propensity weighting leaves the trained model fixed, makes overlap failures visible instead of extrapolating across them, and is checked against ground truth on MoVi, where the regression estimate is a baseline.

Why not match directly on the covariates instead of estimating a propensity score? This is a real threat. CARE-PD's covariate set is small and mostly discrete (severity, medication, freezing), so the dimensionality argument for propensity scores is weak here. We will run coarsened exact matching alongside weighting. The claim of the proposal is balanced evaluation plus part localization, not propensity scores in particular. If exact matching gives the same decomposition with less trimming, we adopt it.

Why not just use a flat encoder with adversarial site removal? Adversarially removing site information from a single embedding does make clinics harder to tell apart. It also removes any genuine population signal correlated with site, such as severity, which balancing is designed to protect. And it gives no location for the shift. The flat-encoder arm of the decomposition, together with a site-adversarial flat baseline under the same budget, tests whether the part split is needed.

Decisive Checks

On MoVi, the method passes if it does three things. Under a pure acquisition shift, the decomposition must attribute at least most of the gap to acquisition. Under a pure BMI and sex shift, it must attribute at least most of the gap to population. And under video versus mocap, the per-part probes must rank the distal limbs and root above the trunk, an expected pattern that the raw jitter and drift statistics should confirm.

On CARE-PD, the headline result is the fraction of each fold's LODO gap explained by patient covariates, with bootstrap intervals and overlap diagnostics. Part-selective adaptation should match or beat MIDA in held-out macro-F1 while beating both the random-branch and unbalanced-localization ablations. A large population term across folds would itself be the finding. It would mean that current cross-site benchmarks for clinical gait partly measure differences in who was enrolled, and that sites should report covariate-balanced scores.