A wearable motion sensor on the lower back records every walk a patient takes for a week. Clinical trials then collapse those recordings into one number per patient, such as average walking speed. This proposal treats each day of wear the way a pathologist treats a needle biopsy: it is a small, biased sample of a heterogeneous whole. The most informative samples (long, uninterrupted walks) are rare. The most common samples (short bursts around the home) are both noisy and the ones the sensor algorithms measure worst. We propose to learn how to weight and summarize walking bouts so that the resulting score is as repeatable as possible from week to week, and as sensitive as possible to real disease change. The key constraint is that the score is fitted without using clinicians' subjective ratings as the training target. The practical output is a method, plus an answer to a question every trial designer asks: how many days of monitoring are needed to reach a given reliability?
The opportunity is concrete. In the Mobilise-D validation, real-world walking speed from a lower-back sensor agreed well with a multi-sensor reference when only correctly matched walking bouts were scored: the intraclass correlation coefficient (ICC), the share of total variance that reflects true differences between people rather than measurement noise, ranged from 0.57 to 0.88. When all detected bouts were aggregated naively, the ICC fell to 0.33 (Mobilise-D insights to estimate real-world walking speed in multiple conditions with a wearable device). In parkinsonian patients, short everyday bouts correlated weakly with clinical scales (|r| 0.2–0.4), while rare bouts longer than 60 seconds correlated much better with postural instability (|r| 0.5–0.6), even though most steps were taken in short bouts (Construct validity of instrumented gait assessments in hospital and daily life mobility in patients with Parkinson's disease and atypical Parkinson's syndromes). Both findings point to one mechanism: the reliability of real-world gait is limited by which bouts get sampled and how well each is measured, not only by the sensor itself.
Core Claim
A patient-week score built as a weighted combination of bout-level quantiles, stratified by bout duration and inverse-weighted by each bout's expected algorithmic error, can reach the week-to-week reliability of a fixed "90th-percentile speed in long bouts" summary with fewer monitoring days, or exceed it at the same number of days. We choose the weights by maximizing a variance-partition criterion: true between-person and between-visit variance relative to within-person sampling variance. We do not regress onto clinician scores. This keeps rater subjectivity out of the definition of the score, and clinician scores serve only as an external check.
Two things distinguish this from the obvious idea at this intersection, which is quantile multiple-instance learning over walking bouts trained to predict disease severity. That idea borrows the quantile pooling layer of Multiple Instance Learning for Heterogeneous Images, which sorts patch-level predictions from a tissue slide into a quantile function and raised histologic subtype accuracy from 89.8% (max pooling) to 95.2%. Trained on clinical scores, however, such a model inherits the "consensus label" problem: it learns to reproduce raters' integer cutoffs. It would also mostly rediscover what DeMMO already found by stability selection, namely that 90th-percentile speed and cadence in longer bouts are the most reliable progression predictors. Our first difference is the training objective: reliability and responsiveness, not agreement with raters. Our second is an explicit per-bout measurement-error model estimated from reference-sensor data. Neither the histopathology work nor DeMMO has an analogue of this second ingredient.
Research Program
The first component is a bout-level error model. The Mobilise-D technical validation data pair each lower-back bout with the INDIP multi-sensor reference across six cohorts (Assessing real-world gait with digital technology?). From these pairs we fit the variance of the walking-speed error as a function of bout duration, estimated speed, turn content, and cohort. This model is expected to capture the documented failure patterns: slow speeds are overestimated, fast speeds are underestimated, stride length has roughly 20% relative error, and the proximal femoral fracture cohort is tracked worst. It acts like a quality score for each biopsy core.
The second component is the aggregation. For patient $i$ in week $w$, we compute error-weighted quantiles $q_{k}(i,w)$ of bout walking speed and cadence within each cell $k$. A cell is a pair of one duration stratum (for example under 30 s, 30–60 s, over 60 s) and one quantile level (10th to 90th). The score is
$$s_{i,w} = \sum_k \alpha_k \, q_k(i,w),$$
where the weights $\alpha_k$ are non-negative and sum to one. The weights are chosen to maximize
$$J(\alpha) = \frac{\sigma^2_{\text{person}}(\alpha) + \lambda\,\sigma^2_{\text{visit-change}}(\alpha)}{\sigma^2_{\text{week}}(\alpha)}.$$
Here $\sigma^2_{\text{person}}$ is the between-patient variance, $\sigma^2_{\text{visit-change}}$ is the variance of true change between clinical visits, $\sigma^2_{\text{week}}$ is the variance across weeks within the same visit window, all estimated from a mixed model on training patients, and $\lambda$ trades off static reliability against responsiveness.
A pure reliability objective would reward traits that are stable but unrelated to disease, such as leg length. For this reason we residualize $s$ on height, age, and sex before computing $J$, and we require $\lambda>0$ so that the score must also track change over time.
The third component is a sampling-adequacy analysis. Given the single-day reliability $\rho_1$ of a score, the Spearman–Brown relation, which follows from averaging independent daily samples, gives the number of days needed to reach a target reliability $\rho^*$:
$$n = \frac{\rho^*(1-\rho_1)}{\rho_1(1-\rho^*)}.$$
We will check this prediction against empirical resampling of days. We expect it to break down when days are not exchangeable, for example weekdays versus weekends or bout-rich versus bout-poor days. Where it breaks, we will report days-to-target by direct resampling instead. Because ICC depends on how heterogeneous the sample is, every ICC will be reported alongside the standard error of measurement and the minimal detectable change in m/s, and results will be broken down by cohort.
Baselines
The naive reference point is the median walking speed over all detected bouts per week, with no duration stratification and no error weighting. This is the analogue of the Mobilise-D "combined evaluation," which reached ICC 0.33 against the reference system (Mobilise-D insights). Note that 0.33 is agreement with a reference, not test-retest reliability. For week-to-week test-retest ICC of real-world gait we found no reported result on Lacuna, so we will estimate it ourselves for every method.
The strongest existing competitor is the fixed DeMMO-style summary: 90th-percentile walking speed and cadence in longer bouts, with the long-bout threshold at 60 seconds following the parkinsonian construct-validity study. DeMMO reports nMSE 0.769 and weighted correlation 0.464 for longitudinal clinical prediction from weekly summaries, beating eight baselines including deep models. It is the strongest competitor because it already encodes the idea that rare long bouts carry the signal. We will run it on the same bouts, weeks, and train/test patient splits as our method, and feed both summaries into the identical DeMMO regression to compare predictive performance.
A third baseline is a learned quantile multiple-instance model over raw bout features in the style of the histopathology work, trained directly on clinical scores with the same patient splits and a matched hyperparameter search budget. Two ablations isolate our ingredients: removing the error weighting (plain quantiles), and fixing $\alpha$ to uniform weights over the long-bout cells.
Beating the strong baseline means one of two outcomes on held-out patients. The first is a higher week-to-week ICC with a bootstrap confidence interval that excludes the P90-long-bout value. The second is the same ICC with at least one fewer monitoring day, while clinical correlation is not worse.
Why not just…?
Why not just use 90th-percentile speed in long bouts, as DeMMO already selected? This is the real threat, and it may well be nearly optimal. The learned weights could simply converge onto those cells. That outcome is informative in itself: it would validate the fixed summary through a criterion that does not depend on raters, and the error model would still supply confidence intervals and days-needed estimates that no current summary provides. The method's claim to add value rests on the error-weighting ablation. If error weighting does not improve reliability, especially in the fracture and Parkinson's cohorts where tracking is worst, we will report that the fixed summary suffices.
Why not just record for more days? Averaging does reduce sampling noise, roughly as one over the number of independent days. Longer wear, however, costs adherence, and the parkinsonian study shows that some patients produce very few long bouts in any given week. The Spearman–Brown analysis directly measures the trade: our method's contribution is expressed in days saved at equal reliability, so the comparison is built into the main result.
Why not just train a deep model end to end on clinical scores? Clinical scales are integers assigned by raters, and discrete grading produces disagreement near category boundaries. About 35% expert disagreement is reported for cataract grading (A Computer-Aided Diagnosis System of Nuclear Cataract via Ranking), and rater reliability depends on how many ordinal categories a scale uses (Comparing the reliability of the single leg squat test). DeMMO also found that deep models underperformed structured linear models on this kind of noisy longitudinal data. We include the deep quantile model as a baseline. If it wins on clinical correlation but loses on week-to-week reliability, that would support our argument that it has fitted rater structure rather than gait.
Why not just use a supervised clinic walk? The 2x10 m walk correlated poorly with balance scores for gait variability (|r| 0.0–0.5) because it samples too few strides, while real-life walking shows larger longitudinal effect sizes in ataxia (Capture of Longitudinal Change in Real-Life Walking in Cerebellar Ataxia). We will still report the 2-minute walk test's correlation as a ceiling reference wherever it was collected.
Proposed Experiments
The error model will be fitted on the Mobilise-D technical validation data (108 participants, 2.5 hours of free living with the INDIP reference) and validated by leave-one-cohort-out. The aggregation weights will be learned and tested on the Mobilise-D longitudinal cohort (about 2,400 participants, five visits over two years). This requires bout-level outputs rather than the weekly summaries DeMMO used, and obtaining them is a data-access dependency we flag explicitly. External checks will use the Mobility_APP seven-day home monitoring data (92 patients with Parkinson's disease, multiple system atrophy, or progressive supranuclear palsy). There we split days into halves for test-retest reliability and correlate scores with UPDRS, the Berg Balance Scale, and PIGD, against the published long-bout reference of |r| 0.5–0.6.
Four checks are decisive. First, held-out week-to-week ICC, standard error of measurement, and minimal detectable change versus the P90-long-bout baseline, reported by cohort. Second, days-to-ICC-0.8 from both Spearman–Brown and direct resampling. Third, standardized response to change between visits. Fourth, whether the residualized score still predicts clinical change through the DeMMO regression at least as well as the baseline, which rules out the trivial solution of rewarding stable but irrelevant traits. A negative result, in which learned weights match the fixed summary and error weighting adds nothing, would be reported as evidence that the current Mobilise-D summary is already sampling-efficient.
References
- Mobilise-D insights to estimate real-world walking speed in multiple conditions with a wearable device
- Construct validity of instrumented gait assessments in hospital and daily life mobility in patients with Parkinson’s disease and atypical Parkinson's syndromes: an exploratory study
- Multiple Instance Learning for Heterogeneous Images: Training a CNN for Histopathology
- DeMMO: Longitudinal and Cross-Disease Modelling of Digital Mobility Outcomes via Multi-Task Learning
- Assessing real-world gait with digital technology? Validation, insights and recommendations from the Mobilise-D consortium
- A Computer-Aided Diagnosis System of Nuclear Cataract via Ranking
- Comparing the reliability of the single leg squat test using two, three, and four category ordinal rating scales
- Capture of Longitudinal Change in Real‐Life Walking in Cerebellar Ataxia Increases Patient Relevance and Effect Size