How Much of a Movement Must a Rater See? Receptive-Field Sweeps for Parkinson's Motor Scoring

Neurologists grade Parkinson's disease by watching a patient tap their fingers or walk, then assigning an integer from 0 to 4 on the Unified Parkinson's Disease Rating Scale (UPDRS). Experts disagree on these integers as often as 53% of the time (TULIP). Several of the features the scale asks raters to judge are properties of the whole trial rather than of any single moment: whether tapping amplitude decays across ten taps, whether rhythm breaks down, whether one side lags the other over a full walk. This proposal tests a simple hypothesis. Raters disagree most on exactly the items whose score depends on long stretches of movement, because a human watching in real time, like a model with a short "receptive field" (the span of input it can see at once), judges from a local window. If the hypothesis holds, a model built to look at the whole trial should agree with raters where they agree with each other, and should be most informative where they diverge.

The deliverable is a measurement tool more than a new architecture. For any movement task, we train severity predictors whose temporal context is capped at $w$ movement cycles, sweep $w$ from one cycle up to the full trial, and record how agreement with each individual rater changes. We fit these predictors against every rater's own labels, with rater-specific score thresholds, instead of against a consensus label. The central question is whether the $w$ at which accuracy stops improving predicts, item by item, how much raters disagree. That links the receptive-field literature to the rater-variability problem in a way that yields a falsifiable claim, not just another accuracy number.

Core Claim

Two claims are new, and both can be checked. The first is that the temporal context a predictor needs for a UPDRS item is measurable, differs across items, and correlates positively with inter-rater disagreement on that item. The second is that a continuous severity score fit to individual raters' labels, where each rater gets their own cutpoints on a shared scale, agrees with a held-out rater at least as well as a second human rater does. It should also reduce the systematic errors that a short-context model makes on trial-level features.

The motivation comes from VisionMD. That study applied sparse principal component analysis to more than 1,200 clinical videos. It found that levodopa-responsive improvement splits into three kinematic dimensions: speed, cycle-to-cycle consistency, and timing. Two of these, consistency and timing variability, are defined only across many cycles. Sequential windowed pipelines cannot see them directly. 1D-convolutional transformer, for example, classifies 100-step gait windows and then takes a majority vote. Frame-level models cannot see them either. The same study found that levodopa mainly changes speed and consistency rather than amplitude, which suggests the scale's emphasis on amplitude may be poorly matched to what actually changes clinically.

Research Program

The model is a latent-severity model with per-rater cutpoints. For trial $i$, a predictor $f_w$ with temporal context $w$ maps the keypoint sequence $x_i$ to a scalar severity $s_i = f_w(x_i)$. Rater $r$'s ordinal label $y_{ir}$ follows

$$ P(y_{ir} \le k) = \sigma(\tau_{r,k} - s_i), $$

where $\sigma$ is the logistic function and $\tau_{r,k}$ is rater $r$'s learned threshold between scores $k$ and $k+1$. Fitting all raters jointly separates disagreement that is a stable offset ("rater B scores everyone higher") from disagreement that concerns the movement itself. The second kind is the target of the receptive-field analysis. When a new rater or the consensus label is the target, we use averaged thresholds.

We use two families of $f_w$, so that the conclusion does not rest on one architecture. The first is a transparent feature model. It computes VisionMD-style kinematics (mean and root-mean-square velocity, per-cycle amplitude, cycle-to-cycle standard deviation, decrement slope, frequency) over windows of $w$ cycles. It pools them over the trial and passes them through the ordinal head. The second is a learned model with factored joint-by-time attention, reusing the channel-split spatial and temporal attention of STCFormer. STCFormer brings full spatio-temporal attention down from $O((TS)^2)$ to $O(T^2S + TS^2)$ for $T$ frames and $S$ joints, which makes whole-trial context affordable. Its local convolutional positional embedding keeps short-range detail. In the learned model, $w$ is enforced by a banded attention mask. STCFormer supplies the method. It was only ever evaluated on pose accuracy (40.5 mm error on Human3.6M), never on clinical scores, and we do not claim its geometric gains carry over.

The analysis step produces a curve $A_j(w)$ for each item $j$: agreement with a held-out rater as a function of context. It also produces a saturation point $w_j^*$, the smallest $w$ within one standard error of the best agreement. We then test whether $w_j^*$ correlates with inter-rater disagreement across items. For spatial context, the same sweep caps attention to the task limb, the same-side body, or the whole body. This speaks directly to SPA-PTA's finding that unrestricted non-local graph attention injects noise from limbs that are not tremoring, which constrains our design. We expect the spatial optimum to be narrow for tremor items and wide for gait and bilateral items. Measuring that difference, rather than assuming one fixed trade-off, is part of the contribution.

Baselines

The naive reference is a predictor that always outputs the training-set majority score for each item, reported next to the observed agreement between rater pairs. The rater-pair agreement is the ceiling that any claim of "objectivity" must be measured against. We report quadratic weighted kappa and exact agreement. No reported kappa was found on Lacuna for TULIP or TIM-TREMOR, so both the baseline and the human ceiling will be computed by us from the raw labels.

The strongest existing method on per-task UPDRS scoring is the TULIP 3D multilayer perceptron. It reports an F1 of 0.69 for 0–4 score prediction, against 0.57 for 2D inputs, and its 3D random forest reaches an F1 of 0.72 on gait (TULIP). We rerun both on the same triangulated keypoints, the same leave-one-subject-out splits, and the same hyperparameter budget as our models. Beating them means higher F1 against consensus labels together with higher kappa against held-out individual raters. On gait-sensor data the strongest competitors are HCT, with 87.10% Hoehn and Yahr staging accuracy on PhysioNet, and 1D-convolutional transformer, with 87.89% staging accuracy using windowed voting. We retrain both under our folds, because fold construction on PhysioNet strongly affects reported accuracy. On TIM-TREMOR the reference is SPA-PTA at 76.4% three-level rating accuracy.

Two ablations isolate the key ingredients. The first replaces the per-rater cutpoint head with a standard classifier trained on consensus labels, which tests the rater-modeling component. The second fixes $w$ at one cycle for the learned model, and at one 100-step window with voting for the gait model, which tests whether whole-trial context matters. Every comparison uses identical inputs, splits, and tuning trials.

Why not just…?

Why not just feed hand-crafted whole-trial features into a random forest? This is the most serious threat. With 15 TULIP subjects it may well match any learned model, and TULIP's own random forest is competitive on gait. For that reason the feature model is a first-class arm of the study, not a straw man. The receptive-field sweep runs on it directly, since windowed features have an explicit $w$. If features win on accuracy, the scientific claim about $w_j^*$ and disagreement still stands. The learned attention model then earns its place only if it beats the features on PhysioNet, which has 166 subjects, or on the VisionMD archive.

Why not just train on the consensus score with label smoothing? Smoothing treats all disagreement as symmetric noise. Per-rater cutpoints separate constant rater offsets from disagreements about the movement, and only the latter can be traced to context length. The consensus-classifier ablation tests this head-on. If it matches per-rater kappa, we will report that rater modeling adds nothing on these data.

Why not just make the CNN deeper or the windows longer? Deeper convolution does widen context. But HCT found standalone 1D-ConvNets lower and less stable across folds (94.5% vs. 97.0% detection). Longer windows are simply points on our $w$ sweep, so the experiment itself answers this question instead of assuming an answer.

Proposed Experiments

The primary test runs on TULIP. The file indicates each subject has 29 UPDRS Part III scores from three independent neurologists, which is what per-rater fitting and held-out-rater evaluation require. We train on two raters and evaluate on the third, rotating through all three, with leave-one-subject-out folds. We compute $A_j(w)$ and $w_j^*$ per item and the rank correlation between $w_j^*$ and pairwise rater disagreement. The hypothesis fails if the correlation is near zero, or if finger tapping and gait saturate at a single cycle. With about 25 tasks the item-level correlation has modest power, so we pre-register its sign.

The second test runs on PhysioNet gait. We sweep context from a single stride through 100-step windows to the full walk, predicting UPDRS where it is available and Hoehn and Yahr stage otherwise. The key question is whether full-walk context removes the Stage 3 errors that HCT attributes to class imbalance. If it does not, and reweighting the loss does, that is a clean negative result for the context explanation.

The third test is objective and does not depend on raters at all. If the VisionMD archive is accessible, the continuous severity $s_i$ should separate ON from OFF levodopa trials within each patient better than predicted integer scores do. This checks whether the score captures real motor change or only reproduces raters' habits.

Across all experiments, the decisive result is a match: held-out-rater kappa at or above rater-pair kappa, together with a positive link between required context and disagreement. That combination would show the model tracks what raters are trying to measure without reproducing the part of their judgment that comes from seeing too little of the movement.