Content-Corrected Speech Rhythm as a Within-Person Biomarker

Speech rhythm metrics describe the timing of a person's speech: what share of time goes to vowels, and how much neighbouring vowel or consonant stretches differ in length. They are cheap, interpretable, and have separated Parkinson's patients from controls. They have not held up as cross-sectional markers of cognitive decline. The most direct test, on 66 Italian speakers, found no group difference on any of ten metrics (Automatic Detection of Rhythmic Features in Pathological Speech of MCI and Dementia Patients). Research on blood and physiological biomarkers suggests a way out. A quantity that is useless as a one-time snapshot can become informative when each person is compared against their own history, because between-person spread and moment-to-moment noise stop swamping the signal. This is the same logic by which serial albumin outpredicts admission albumin (Association between albumin changes and prognosis in older patients with acute myocardial infarction).

Moving rhythm metrics from snapshots to trajectories is not enough, though, because rhythm metrics carry a large noise source that blood assays lack. Their values depend heavily on the words spoken, since syllable structure largely determines interval durations (Is speech rhythm an intrinsic property of language?). A person who says "strengths" in one session and "banana" in the next will show a rhythm "change" that has nothing to do with their brain. This note proposes to subtract, session by session, the rhythm that the spoken phoneme sequence alone would predict, and to track only the leftover personal timing deviation over months. The claim is that this correction is what turns rhythm into a usable within-person biomarker, and that its benefit should grow with how much the spoken content varies between sessions.

Core Claim

A speaker's rhythm residual is the gap between their observed interval timing and the timing a reference duration model predicts for the same phoneme string at the same speaking rate. It has much lower session-to-session variance than the raw metric. Its per-person slope therefore separates declining from stable speakers with fewer sessions than the raw metric's slope does. Formally, for session $s$ of person $j$, let $m_{js}$ be a raw rhythm metric such as vowel percentage (%V) or the normalized pairwise variability index (nPVI, the mean rate-normalized difference between adjacent vowel durations). Let $\hat m(\mathbf{p}_{js}, r_{js})$ be the value a population duration model predicts from the aligned phoneme sequence $\mathbf{p}_{js}$ and articulation rate $r_{js}$. The residual $e_{js} = m_{js} - \hat m(\mathbf{p}_{js}, r_{js})$ is then modelled as

$$e_{js} = a_j + b_j t_{js} + \varepsilon_{js},$$

where $t_{js}$ is time since the first session, $a_j$ and $b_j$ are the person's own intercept and slope (random effects in a mixed model), and $\varepsilon_{js}$ is within-person noise. The biomarker is $b_j$. Slope precision scales with $\sigma_\varepsilon^2$ divided by the spread of session times. So any reduction in within-person noise that the content correction buys translates directly into fewer sessions needed to detect a given rate of decline.

The contrast with prior work is specific. Exploiting Longitudinal Speech Sessions via Voice Assistant Systems for Early Detection of Cognitive Decline showed that adding a person's earlier sessions lifts acoustic MCI detection F1 from 58.6% to 71.2%, which motivates the longitudinal framing. But it used self-supervised speech embeddings (HuBERT, WavLM, XLS-R) that entangle speaker, channel, and content. Belmonte et al. computed exactly the metrics we use, but only cross-sectionally, and with a landmark segmenter whose boundary F1 on the CLIPS corpus was 0.415. This proposal does neither. It tracks explicit metrics within person, and it explicitly removes the content component, which neither line of work attempts.

Research Program

The work has three parts.

The first part is segmentation that is good enough to trust. We replace acoustic-landmark detection with forced alignment, meaning an ASR acoustic model snaps a known or recognized phone sequence to the audio. We use hybrid alignment models of the kind built for aging conversational speech in Automatic Speech Recognition for ILSE-Interviews. Vowel and consonant interval boundaries are read off the alignment. We validate boundary precision, recall, and F1 on CLIPS under the same tolerance Belmonte et al. used, so the 0.415 figure is a direct reference point. To limit damage from residual misalignments, we also compute the median-based pairwise variability index of An improved pair-wise variability index, which that paper found more sensitive to the full-versus-reduced vowel contrast (31.6% versus 17.4% difference).

The second part is the content model. For each language we fit a duration model on healthy normative speech. It predicts each interval's duration from phone identity, the number of phones in the interval (the idea behind the segment-count normalization in the control/compensation index of Chinese and Italian speech rhythm: normalization and the CCI algorithm), position, and local articulation rate. We then aggregate the predictions into expected %V, variability of consonant durations, Varco (rate-normalized coefficient of variation), and nPVI. For English, the normative source is the Voiceome cohort (A Longitudinal Normative Dataset and Protocol for Speech and Language Biomarker Research), with 6,650 participants and 12 tasks. Its 1,382 people with repeat sessions roughly three weeks apart also give a healthy test-retest noise floor. From that floor we compute the minimal detectable change, $1.96\sqrt{2}\,\sigma_\varepsilon$, which comes from the spread of a difference between two independent noisy measurements. This tells a clinician how large a rhythm shift must be before it is more than noise.

The third part is trajectory modelling. It uses the mixed model above, with irregular session spacing handled natively through $t_{js}$ rather than by resampling. This matters because Voiceome retest gaps vary as $22.80 \pm 20.91$ days. Person-level slopes, alone or alongside intercepts, feed simple classifiers for decline and change.

Baselines

The naive reference point is the single-session raw rhythm metric, meaning the Belmonte-style snapshot, classified with the same logistic regression we use elsewhere. Its expected performance is near chance on cognitive groups, given that paper's null result. Any method that does not clearly beat it has not shown that rhythm carries information.

The strongest existing method on longitudinal home recordings is the acoustic self-supervised pipeline with session history from Qi et al. It reports an acoustic F1 of 71.2% for MCI-versus-control detection with history, and 73.7% F1 for session-to-session change prediction on the voice-assistant dataset. We will rerun it on the same speakers, splits, and sessions, with the same hyperparameter search budget as our models. We report both our rerun and their published numbers so that any reproduction gap is visible. Beating it means matching or exceeding its F1 with a feature set of under twenty interpretable numbers per session. Alternatively, it means a significant gain when our residual slopes are added to its embeddings in late fusion.

On the two-visit ADReSSo progression task, the reference numbers are the challenge baseline of 68.75% accuracy and 66.67% mean F1 (Detecting Cognitive Decline Using Speech Only: The ADReSSo Challenge). The strongest reported system is the text-based MUET-RMIT system at 73.80% average F1 (Tackling the ADRESSO Challenge 2021: The MUET-RMIT System). We use the official splits and evaluate once on the test set.

Two further baselines isolate the proposal's ingredients. The key ablation tracks raw rhythm metrics longitudinally with the identical mixed model, differing only in the absence of content correction; this isolates the correction itself. A second ablation tracks global speech rate and pause rate trajectories with the same model. This tests whether segmental timing adds anything beyond tempo.

Why not just…?

Why not just use self-supervised embeddings with session history, as Qi et al. already did? This is the real threat. On in-home data from fixed devices, embeddings may simply win on F1. The case against relying on them alone is fragility and non-specificity. Motor, Cognitive, or Corpus? found such embeddings lose 12.5 balanced-accuracy points under recording-condition shifts. It also found they could not distinguish Parkinson's from dementia patients in the TREND cohort, which suggests they pick up general illness or channel cues. We settle this empirically in two ways. First, we compare both methods under deliberately perturbed channel conditions: re-encoded audio, added room noise, and a changed microphone simulation between early and late sessions. Second, we test whether our residual slopes add to embeddings in fusion. If embeddings win clean, win under perturbation, and gain nothing from fusion, then the accuracy claim fails. What would remain is the reliability and interpretability result.

Why not just track global speech rate, which needs no segmentation? In Italian Parkinson's speech, %V separated patients from controls strongly ($p = 9.28 \times 10^{-10}$) and correlated with disease duration ($r = 0.54$). Articulation rate measured by vowel-onset intervals did not reach significance ($p = 0.09$) (Speech Rhythm in Parkinson's Disease: A Study on Italian). Adding segmental metrics to rate and pitch also improved dysarthria severity classification by 15% relative (Dysarthria Detection and Severity Assessment Using Rhythm-Based Metrics). Both results are cross-sectional and motor rather than cognitive, so the speech-rate trajectory baseline above is the actual test.

Why not just track raw rhythm metrics over time, without the content model? This is the obvious version of the idea, and it may already beat the snapshot. Our claim is that it leaves avoidable noise in whenever sessions differ in what is said. The raw-trajectory ablation measures exactly this. We also predict where the correction should matter little: tasks with fixed content, such as repeated picture descriptions, where the phoneme string barely changes. If the correction helps equally on fixed and variable content, our explanation is wrong even if the numbers improve.

Why not just use transcripts and a language model? Transcript models are strong on ADReSSo (73.80% F1). But ASR on this speech had a word error rate near 60%, and Qi et al. found acoustic features alone predicted session-to-session change at 73.7% F1. We treat text as complementary and report fusion. We also note honestly that our content model depends on recognized phone strings for unscripted tasks, so we measure how residual reliability degrades as simulated recognition errors increase.

Proposed Experiments

The first decisive check is reliability. On Voiceome repeat sessions, we compare within-person intraclass correlation and minimal detectable change for raw versus content-corrected metrics, stratified by task type. The prediction is a clear reliability gain on free-content tasks and a negligible gain on fixed-text reading. There are no reported acoustic reliability numbers on Lacuna for Voiceome to compare against, so both conditions are computed fresh.

The second check is detection of decline on the voice-assistant cohort: 35 people, 7 quarterly sessions, and tasks such as verbal fluency whose content varies between sessions. We compare residual slopes against raw slopes, speech-rate slopes, and the history-based embedding pipeline on MCI-versus-control detection and change prediction, using leave-one-subject-out evaluation. With 35 people we report bootstrap confidence intervals rather than point wins. Access to this dataset depends on the authors sharing it. If it is unavailable, the German ILSE interviews, which span 20 years, substitute, with cognitive status from the study's assessments.

The third check is ADReSSo progression. Here only two visits exist and the picture is identical. We expect the content correction to add little over raw metrics, and a large gain would count against our mechanism. The segmentation check on CLIPS runs first, because if forced alignment does not clearly exceed 0.415 boundary F1, the downstream null results of Belmonte et al. cannot be reinterpreted.

The final check is specificity, and it is necessarily exploratory. On Parkinson's corpora we test whether %V residual slopes track motor severity. On cognitive cohorts we test whether nPVI residuals behave differently. The question is whether interpretable timing measures avoid the Parkinson's-versus-dementia confusion reported for embeddings. No longitudinal cohort with both diagnoses exists in the file, so any positive finding here is a hypothesis for future data collection rather than a conclusion.