Tablet-based cognitive tests can now screen older adults for dementia in under five minutes, often with no clinician in the room. That removes the specialist bottleneck. It also creates two quieter problems. First, the groups these tests most need to serve, such as people with little schooling, minority-language speakers, and people unfamiliar with touchscreens, are usually a small slice of the training data, so any model fitted to them is noisy. Second, the signals that make these tests work best are pauses, restarts, abandoned attempts, and slow strokes. These are exactly the behaviours that learning-analytics researchers read as disengagement in online courses. An 80-year-old who hesitates may be cognitively impaired, or may simply never have held a stylus. How much weight a pause deserves could plausibly differ between groups, while the meaning of a tremor-free, fluent stroke probably does not.
We propose an interpretable screening model built from one learned curve per feature. For each feature, the model decides separately, with a formal test, whether a small subgroup can share the curve estimated on the full cohort or needs its own. The idea comes from multi-ancestry genetics. There, methods such as LOG-TRAM borrow information from a large European cohort only in genome segments where local architecture is shared with the smaller target cohort. We carry that selective borrowing over from genome segments to behavioural features, and we test a specific hypothesis. We expect curves for motor and kinematic features to be shared across groups. We expect curves for engagement features, such as pauses, retries and task abandonment, to be where subgroup differences live.
Core Claim
A pooled screening model gives small subgroups stable but possibly biased curves. A separate model per subgroup gives them unbiased but overfit curves. Borrowing strength one feature at a time, with FDR control over which features are declared different, should do better than either on within-subgroup calibration and false-positive rate. It should also produce an auditable statement that clinicians can read: "for low-education users, pause duration is weighted differently; stroke velocity is not." FDR control means bounding the expected fraction of "this feature differs" claims that are false.
What is new relative to the closest work is the combination. Selective, heterogeneity-tested borrowing has not been applied to behavioural or digital-biomarker features. Within-test disengagement signals have not been treated as a distinct feature family whose subgroup-specific meaning is itself a scientific finding.
Grounding in Prior Work
Two papers establish the opportunity and the gap. PENSIEVE-AI is a self-administered four-task drawing test scored by a VGG-16 image network, a convolutional network originally designed for photo classification. It reached an AUC of 93.2% in 1,758 multiethnic Singaporean older adults. Adding temporal "drawing activities" such as pauses, stroke intervals and attempts raised the AUC from 88.2%, and about 70% of participants finished unsupervised. The model is a black box, and it handles literacy only by including years of education as an input. Li et al.'s character-writing study found that three process features alone reached an AUC of 0.918 in 72 people: average pause time, number of pauses and writing-speed variability. Both papers motivate our focus on pause-type features. The Li study also shows what a tiny cohort looks like, and its wide confidence interval (0.854–0.982) is the variance problem we target.
The online-attrition literature supplies the reinterpretation of those same pause features. Mogavi et al. showed that raw activity counts conflate opposite behaviours: badge-gamers and challenge-seekers can look identical in counts. They showed that attendance gaps and endurance, decoded into latent engagement states, predict dropout. Dass et al. found short-window moving averages of progress to be the dominant dropout signal. We take only one lesson from this work, and it constrains our design: gap-and-retry behaviour is ambiguous between low motivation and low ability. We do not reuse their predictors. We reuse their feature families: inter-task gaps, retry counts, abandonment, and short-window progress slopes within a session.
The interpretability literature supplies the model class. RAITR fits a sparse linear model first and admits a nonlinear spline term only when it earns its place under a concordance-based information criterion. That work also reports that ordinary cross-validation over-selects nonlinear terms. We reuse this "reluctant" admission rule for our subgroup correction terms. The file also notes that RAITR degrades at small sample sizes, which is exactly why subgroups need borrowed strength. Haddada et al. supply our interpretability metric: the Kendall correlation between a model's feature ranking and clinicians' ranking. They report 1.0 for tree models and −0.39 for nearest-neighbour models on OASIS-3. PECLIDES Neuro shows that pruned rule sets can keep 94.7% accuracy. It stands as an alternative interpretable form we compare against, not one we adopt.
The borrowing mechanism comes from genetics. LOG-TRAM estimates local genetic correlation in 1-megabase segments and borrows from UK Biobank only where it is high, raising Biobank Japan's effective sample size by 49%. hetFDR tests for ancestry differences in effect, using the overall mean effect as a weight on the heterogeneity p-values under FDR control. We borrow the logic of both methods but not their machinery. Both operate on summary statistics under a genetic model, and neither has been applied to behavioural features.
Research Program
Let $x_j$ be feature $j$, which is either a drawing-kinematic or a session-engagement feature. The reference model, fitted on the full cohort, is additive: $\text{logit}\,P(\text{impaired}\mid x)=\sum_j f_j(x_j)$. For subgroup $g$ we write
$$ f_j^{g}(x_j) = f_j(x_j) + w_j^{g}\,\delta_j^{g}(x_j), $$
where $\delta_j^{g}$ is a subgroup-specific correction curve and $w_j^{g}\in[0,1]$ is a borrowing weight. Estimation proceeds in three steps.
First, for each feature and subgroup, we compute a heterogeneity p-value for $\delta_j^{g}=0$ by a within-subgroup permutation or score test. We weight these p-values by $|f_j|$, following hetFDR, and select the features whose corrections survive at 10% FDR.
Second, for surviving features only, we fit $\delta_j^{g}$ reluctantly, in the RAITR sense: a linear correction first, with a spline correction admitted only if it improves held-out concordance. The weight $w_j^{g}$ is set from the estimated correlation between subgroup and reference curves, by analogy to LOG-TRAM's local co-heritability. All non-surviving features have $w_j^{g}=0$ and use the full-cohort curve.
Third, the output for each subgroup is a small set of two-dimensional shape plots, one per feature, with flagged corrections overlaid. A clinician can read these directly.
Features come in two blocks, and every analysis is run separately by block. The kinematic block holds stroke velocity, pressure and speed variability. The engagement block holds between-stroke and between-task pauses, number of attempts, restarts, tasks abandoned before completion, requests for help, and short-window progress slopes within the session. Incomplete tasks are encoded as informative events, not dropped as missing data.
Baselines
The naive reference point is the pooled additive model with no subgroup corrections. It has the same features and the same reluctant spline selection. Beating it means lower within-subgroup calibration error and a lower false-positive rate at fixed sensitivity, in subgroups where it is miscalibrated.
The opposite extreme is a separately fitted additive model per subgroup. It isolates the variance cost that borrowing is meant to avoid. We expect it to lose most at the smallest subgroup sizes.
The strongest predictive competitor is the PENSIEVE-AI deep model, which reports an AUC of 93.2% overall and 88.2% without drawing activities on its hold-out set (PENSIEVE-AI). No subgroup-specific AUCs were found on Lacuna for PENSIEVE-AI. The paper states only comparability across education levels. We will re-train this architecture on identical splits, with the same tuning budget (an equal number of hyperparameter trials) and years of education as an input, as in the original. We will then report per-subgroup AUC and calibration. We do not need to beat it on overall AUC. The bar is to come within about two AUC points overall while beating it on low-education calibration and false-positive rate. On the small character-writing data, the reference number is the three-biomarker AUC of 0.918 (Li et al.).
The most dangerous baseline is a Bayesian hierarchical additive model with a separate heterogeneity variance per feature. It performs partial pooling that differs by feature without any explicit test. This design could achieve most of our gains. We include it with matched compute.
The ablations isolate our ingredients. One replaces feature-wise borrowing with a single global borrowing weight. One drops the FDR step and admits every correction. One removes the engagement block entirely.
Why not just…?
Why not just use a pooled model with education as a covariate? This is what PENSIEVE-AI does. It shifts risk by education level but cannot change the shape of a feature's curve by group. The pause-type effect we hypothesise is a shape difference. The pooled baseline will show whether this matters. If the pooled model is already calibrated within every subgroup, the proposal's premise fails, and we will report that.
Why not just fit a hierarchical model with per-feature partial pooling? This is a real threat, because it is the same statistical idea without the explicit test. Two things could differ. The first is the auditable product: a list of heterogeneous features with an FDR guarantee, where a hierarchical model yields posterior variances whose "difference" calls lack error control. The second is behaviour at very small subgroup sizes, where the hierarchical variance components are poorly identified. If the hierarchical model matches our accuracy and calibration and its credible-interval calls reproduce our flagged features, we will adopt it as the estimator and keep only the scientific finding about which feature families differ.
Why not just recruit more participants from small subgroups? Dedicated cohorts do reveal new biology, as with Petty et al. and their 63,184 Hispanic/Latino participants. But the whole point of the screening setting is scarcity of clinical assessment. LOG-TRAM's 49% effective-sample gain came without new recruitment. Our subsampling experiment measures how many extra participants selective borrowing is worth.
Why not just explain the deep model with SHAP? Post-hoc attributions can look plausible without reflecting the computation, and they diverged sharply from clinical logic for some models in Haddada et al.. We will compute SHAP-derived rankings for the re-trained deep model and for our model, and compare both against clinician rankings using Kendall's tau.
Proposed Experiments
The primary dataset is the Singapore multiethnic PENSIEVE-AI cohort, requested through its authors. We would use its ~40/20/40 split. Subgroups are defined by ethnicity (Chinese, Malay, Indian) and education (for example, at most six years versus more). The secondary dataset is the 72-person character-writing cohort, for the extreme small-sample regime. The UCI Parkinson's spiral-drawing data serves as a public check on kinematic curve estimation. If cohort access fails, the method and simulation results still stand, but the substantive hypothesis would go untested.
The first decisive check is a subsampling curve. We subsample each minority subgroup's training data to 25, 50, 100 and 200 participants, keeping the test set fixed, and plot within-subgroup calibration error and false-positive rate at 90% sensitivity for all methods. Selective borrowing should dominate the separate-model baseline at small sizes and the pooled model at large sizes. If it never beats the better of the two by a margin outside bootstrap intervals, the idea fails.
The second check is the substantive hypothesis. Among features flagged heterogeneous, engagement features should be over-represented relative to kinematic features. We test this with a permutation test on block membership. If flags are uniformly spread, or concentrated in kinematics, the "attrition signals carry group-specific meaning" claim is refuted.
The third check validates the FDR guarantee in a semi-synthetic setting. On the real cohort, we plant known curve differences into chosen features for one subgroup, and we permute subgroup labels to create a null. We then measure the realised false discovery rate and power of the heterogeneity flags against the per-feature hierarchical model.
Finally, two practising clinicians rank feature importance per subgroup before seeing any model output. We report Kendall's tau against each model, following Haddada et al., along with the number of shape plots a reader must inspect.
References
- PENSIEVE-AI a brief cognitive test to detect cognitive impairment across diverse literacy
- A new method for community-based intelligent screening of early Alzheimer's disease populations based on digital biomarkers of the writing process
- Characterizing Student Engagement Moods for Dropout Prediction in Question Pool Websites
- Predicting Student Dropout in Self-Paced MOOC Course Using Random Forest Model
- A reluctant additive model framework for interpretable nonlinear individualized treatment rules
- Assessing the Interpretability of Machine Learning Models in Early Detection of Alzheimer's Disease
- PECLIDES Neuro: A Personalisable Clinical Decision Support System for Neurological Diseases
- Leveraging the local genetic structure for trans-ancestry association mapping
- Heterogeneous associations between interleukin-6 receptor variants and phenotypes across ancestries and implications for therapy
- Large-scale multi-omics analyses in Hispanic/Latino populations identify genes for cardiometabolic traits