When people avoid hospitals, whether from fear of infection during an epidemic or for any other reason, the patients who still show up are the sickest ones. Stroke admissions in Qatar fell by roughly 40% during pandemic peaks, and the missing patients were mostly mild cases and "mimics" (conditions that look like stroke but are not), so the cases that were recorded skewed severe (Al Jerdi et al., Kamran et al.). Any clinical speech model trained on hospital-recruited recordings inherits this skew. It learns what severe dysarthria sounds like (dysarthria is slurred or weak speech caused by damage to the nerves and muscles of speech). It then fails exactly where remote monitoring is supposed to help: at home, on early and mild decline, and in languages that have almost no clinical speech data.
We propose to stop learning dysarthria from patients and instead learn healthy articulation from healthy people. Healthy tongue and lip motion has been recorded with electromagnetic articulography (EMA: small sensor coils glued to the tongue and lips, tracked in a magnetic field) in several languages. A model that recovers those movements from audio alone, called acoustic-to-articulatory inversion, can turn any smartphone recording into estimated tongue trajectories. Our claim is that a patient's distance from the healthy distribution of articulatory movement parameters is a severity score that does not depend on which patients happened to present to a clinic. It should therefore hold up under avoidance-driven selection. It should also carry over to a low-resource language such as Tamil better than a classifier trained on selected patients, and better than a normative model built in raw self-supervised speech-feature space.
Core Claim
Two kinds of work already exist nearby, and neither covers this. Direct kinematic measurement predicts clinical decline: in ALS, tongue and lip motion tracked with physical sensors, combined with acoustics, predicted the ALSFRS-R bulbar subscore (a clinical rating of speech and swallowing function) with RMSE 1.78 and r = 0.64, and tongue sensors beat lip sensors (Speech-based Estimation of Bulbar Regression in ALS). In spastic cerebral palsy, reduced tongue opening displacement, lower peak velocity, and a reversed timing between peak tongue acceleration and voicing onset (the "AccLag" measure, reaching −46.99 ms in one severe speaker) separated patients from controls (Kinematic analysis of tongue movement control in spastic dysarthria). Both studies required sensors in the mouth, used English only, and had very few speakers. On the inversion side, English-trained inversion on multilingual self-supervised features (MMS-1b) reached a Pearson correlation of 0.564 on held-out Dutch speakers, against 0.330 for classical spectral features (Exploring Self-Supervised Speech Representations for Cross-lingual AAI). That result covered only healthy speakers and involved no clinical task.
The obvious combination would train an inversion model, extract kinematic features, and train a severity classifier on hospital patients. We deliberately do not propose that. A classifier of that kind still learns from the selected, severe cohort, so it inherits the severity paradox. What is new here is the scoring rule. Severity is measured as deviation from a cross-lingual normative model of healthy articulatory movement parameters, fit only on healthy EMA speakers. Pathological labels are used only for evaluation and calibration, never to define the score. A companion "representativeness" check, adapted from Target Population Identification in Sample Selection Bias in Machine Learning for Healthcare, routes recordings that fall outside where the inversion model is trustworthy to a clinician instead of scoring them. That paper worked only on tabular data. We apply its idea to speech, and we define representativeness in articulatory space rather than in raw input space.
Research Program
The first component is a cross-lingual, cross-hardware inversion model. It takes intermediate-layer MMS features, following the finding that layers around two-thirds depth carry the most articulatory information and that fine-tuning for speech recognition hurts inversion (Exploring Self-Supervised Speech Representations for Cross-lingual AAI). It predicts a shared reduced set of tongue tip, tongue body, lip, and jaw trajectories. For architecture we borrow chunked autoregression and speaker embeddings from Speaker-Independent Acoustic-to-Articulatory Speech Inversion, which reported r = 0.784 on English. Training pools HPRC, MOCHA-TIMIT, EMA-MAE, and the Finnish–Russian FROST-EMA corpus (FROST-EMA). Different EMA machines use different sensor layouts and coordinate systems, so each corpus maps into the shared articulator set through a per-corpus output head plus a head-corrected coordinate frame.
The second component turns each estimated trajectory into movement parameters per gesture. For each consonant-vowel opening movement we compute displacement, peak velocity, the velocity-to-displacement ratio (a standard proxy for gestural stiffness in the damped-oscillator view of speech gestures), and an acoustic-anchored timing term analogous to AccLag, measured from the estimated peak tongue acceleration to voicing onset detected in the audio. The normative model is a density over these parameters, conditioned on a broad, language-general gesture class $c$ (for example, tongue-tip closure or lip closure). The class is identified from the trajectory itself, so no target-language phoneme recognizer is needed. That matters because such recognizers are fragile in low-resource languages and in dysarthric speech. The recording-level score is
$$ S = \frac{1}{N}\sum_{i=1}^{N} (\theta_i - \mu_{c_i})^\top \Sigma_{c_i}^{-1} (\theta_i - \mu_{c_i}), $$
where $\theta_i$ is the parameter vector of the $i$-th of $N$ detected gestures, and $\mu_c$ and $\Sigma_c$ are the healthy mean and covariance for class $c$, pooled across speakers of English, Finnish, Russian, and Dutch.
The third component is the representativeness gate. A small network, trained only on healthy EMA speakers, predicts the inversion model's own error from its internal features. Recordings with high predicted error, or whose inferred gestures fall outside the support of the training languages, are referred to a clinician instead of scored. The gate asks "can we measure this speaker reliably?", not "does this speaker look like a hospital patient?", so it does not reintroduce the selection bias.
Baselines
The naive reference point is the Percentage of Total correct Phonemes, the single acoustic feature that Cross-lingual Dysarthria Severity Classification for English, Korean, and Tamil found useful in all three of its languages. We use it together with that paper's full hybrid XGBoost model, which reported an overall F1 of 67.14% (against 64.52% for the intersection feature set and 66.74% for the union) on TORGO, QoLT, and SSNCE with unselected training data. We rerun it on our selected-training protocol with the same splits. Beating it means higher F1 and higher mild-case sensitivity on the unselected test population.
The strongest learned competitor is a supervised classifier or regressor on the same intermediate MMS layer, fine-tuned on the selected hospital cohort across all three languages. It uses the same backbone, the same number of trainable parameters as our inversion head plus scorer, and the same hyperparameter search budget, with selection done on a validation split drawn from the selected cohort, as a real deployment would. Beating it means a smaller performance drop from the selected validation set to the unselected test set, and at least equal performance on the severe cases.
For selection-bias correction we run inverse-propensity weighting and the T-Net and MT-Net target-population models from Sample Selection Bias in Machine Learning for Healthcare, all on the same MMS features. That paper reports that ignoring selection costs up to 22% AUC and that classical reweighting often loses to the naive model. We expect to reproduce the qualitative pattern, but no reported result was found on Lacuna for these methods on speech.
The ablation that isolates our key ingredient keeps the normative-deviation rule and the gate but computes them directly in MMS feature space, skipping inversion. On TORGO, which has real EMA for dysarthric speakers, we also compute the score from measured rather than estimated trajectories. This gives an upper bound and tells us how much the inversion step costs.
Why not just…?
Why not just fine-tune the speech model on severity labels and skip inversion? This is the most serious threat. Self-supervised features very likely contain everything the estimated kinematics contain. Nothing in the file compares self-supervised features against inverted kinematics for clinical prediction: the ALS result compares spectral features against physically measured motion, not against a strong learned acoustic model. We do not claim the inversion step improves accuracy when training and test data match. Our claim is narrower: when training data lacks mild cases, a supervised model has nothing to learn "mild" from, while a model that measures distance from healthy speech does not need mild cases at all. If the fine-tuned model matches us on mild-case sensitivity under simulated selection, the claim fails, and we will report that.
Why not build the healthy-norm model directly on self-supervised features? This is the ablation in the Baselines section. Our reason to expect it to lose is that raw feature space encodes language, accent, and recording channel alongside motor control. Its distance-from-healthy score should therefore flag a new language or a foreign accent as abnormal, the speech equivalent of a stroke mimic. Movement parameters such as stiffness and timing are closer to physical quantities and should shift less across languages. FROST-EMA, which contains native, non-native, and deliberately imitated accents from the same healthy speakers, directly tests this.
Why not just reweight the hospital data toward the home population? Avoidance removes mild cases almost entirely, so there is little overlap to reweight. Sample Selection Bias in Machine Learning for Healthcare already found distribution-alignment corrections underperforming naive models. We run these corrections anyway as baselines rather than assume they fail.
Why not track the lips with a phone camera? Cameras cannot see the tongue, and tongue motion was the more predictive signal for ALS decline (Speech-based Estimation of Bulbar Regression in ALS). A video lip-only variant of our scorer would be a useful future comparison, but none of the benchmark corpora here provides synchronized video, so we do not claim to settle this question.
Proposed Experiments
We first validate inversion where ground truth exists. We report Pearson correlation and RMSE on held-out HPRC and MOCHA-TIMIT speakers, zero-shot on the Dutch corpus against the 0.564 reference, and, most importantly, on TORGO's dysarthric speakers. On TORGO we also check whether estimated displacement, velocity, and timing parameters preserve the patient-versus-control ordering that the measured EMA shows. If inversion flattens dysarthric motion toward healthy motion, which is plausible because it is trained only on healthy speech, the deviation score will underestimate severity. This check alone can kill the approach.
Next we simulate avoidance on TORGO, QoLT, and SSNCE, following the semi-synthetic design of the selection-bias paper. The training and validation pool keeps all severe speakers and progressively drops mild speakers, ending near the roughly 40% reduction concentrated in mild cases reported for stroke. The test set is the full unselected, speaker-disjoint population. We report AUC for any-dysarthria detection, Spearman correlation with severity labels, macro-F1 over severity classes, and sensitivity on mild speakers specifically, all as a function of selection strength. Tamil is held out entirely from the training and validation pool in a leave-one-language-out variant, so that no Tamil pathological labels touch training. This is the low-resource condition that matters.
The decisive checks are three. First, as selection strengthens, the supervised MMS classifier's mild-case sensitivity should fall while ours stays roughly flat. Second, the kinematic normative score should raise fewer false alarms than the feature-space normative score on healthy FROST-EMA non-native and imitated-accent speech and on Dutch controls. Third, the representativeness gate should raise accuracy on the recordings it keeps, and its referral rate should concentrate on recordings where inversion error is measurably high on TORGO. If any of the first two fails, the proposal reduces to "use MMS features," which would itself be a useful negative result.
References
- Impact of COVID-19 pandemic on stroke admissions in Qatar
- Trends in stroke admissions before, during and post-peak of the COVID-19 pandemic: A one-year experience from the Qatar stroke database
- Speech-based Estimation of Bulbar Regression in Amyotrophic Lateral Sclerosis
- Kinematic analysis of tongue movement control in spastic dysarthria
- Exploring Self-Supervised Speech Representations for Cross-lingual Acoustic-to-Articulatory Inversion
- Sample Selection Bias in Machine Learning for Healthcare
- Speaker-Independent Acoustic-to-Articulatory Speech Inversion
- FROST-EMA: Finnish and Russian Oral Speech Dataset of Electromagnetic Articulography Measurements with L1, L2 and Imitated L2 Accents
- Cross-lingual Dysarthria Severity Classification for English, Korean, and Tamil