Separating Planning from Execution in Mandarin Tone Sandhi for Clinical Speech Measures

When a Mandarin speaker says two low "third tones" in a row, the first one usually rises and sounds like a "second tone." This alternation is called third-tone sandhi. Textbooks describe it as a clean rule, but large healthy-speech corpora show it is not clean. The rise is real yet smaller than a genuine second tone, and its size varies with speaker, word frequency and region. That makes sandhi an unusual window for clinical speech science. A patient's speech can fail in two different ways. The patient can fail to plan the alternation, so no rise appears even though the rule calls for one. Or the patient can plan it and execute it weakly, so the rise appears but is compressed. These two failures point to different deficits: the first to a linguistic or planning breakdown, the second to a neuromotor or vocal-fold problem. Today's clinical speech tools for Mandarin cannot tell them apart.

We propose a pitch-based measure that decomposes each speaker's sandhi productions into two parts. The first is how often the planned alternation is applied, a categorical quantity. The second is how far it is realized when applied, a gradient quantity. Both are scaled against that same speaker's own lexical second tones and unaltered third tones. The within-speaker scaling is the core design choice, and it has a clinical payoff. Microphones, rooms and general voice quality affect a speaker's lexical tones and sandhi tones alike, so these effects largely cancel. That protects the measure against the recording-condition shortcuts that inflate accuracy in small clinical speech studies.

Core Claim

A speaker's lexical rising tone and unaltered (non-final) third tone define a personal pitch scale. Sandhi tokens placed on that scale reveal two separable quantities: an application rate and a realized-rise magnitude. A third quantity, the length of the scale itself compared with healthy norms, indexes general laryngeal execution. Together these form an interpretable intermediate measure that can pass the verification, analytical-validation and clinical-validation stages (the "V3" pipeline) advocated by Responsible development of clinical speech AI and A Tutorial on Clinical Speech AI Development.

We make one testable prediction. Peripheral vocal-fold disease should shrink the scale and the realized rise while leaving the application rate at healthy levels. A deficit in planning should lower the application rate even when the scale is intact.

Where This Sits in Prior Work

Three healthy-speech studies motivate the gradient half of the measure. A corpus study of the 3rd tone sandhi in standard Chinese found that sandhi rises differ from lexical rises in peak height and timing. Mandarin Tone Sandhi Realization: Evidence from Large Speech Corpora showed across roughly 2,300 speakers that sandhi syllables have a smaller log-range rise than lexical second tones, with demographic shifts. Improved Contextualized Speech Representations for Tonal Analysis found that sandhi tokens sit on a continuum in fine-tuned wav2vec 2.0 space. Understanding phonology by phonetic implementation supplies the theory: sandhi is reduced effortful execution. All four studies use healthy speakers only. None of them asks whether some tokens are categorical non-applications mixed in with gradient ones.

Gestural composition work supplies the method for that question. Articulatory Characteristics of Icelandic Voiced Fricative Lenition used mixture models on tongue measurements to show that mostly gradient undershoot coexists with about 8% outright gesture absence. Systematic variation in the articulation of the Korean liquid showed that an apparently reduced variant is in fact a deleted gesture. Functional data analysis as a tool for analyzing speech dynamics showed that summarizing whole contours with principal-component curves preserves residual gestures that scalar summaries erase. We carry this undershoot-versus-deletion logic from tongue gestures to the laryngeal pitch gesture of sandhi. The Icelandic result also constrains us: surface acoustics can track articulation poorly. We therefore treat pitch as the observable closest to laryngeal tension, rather than a weak proxy derived through some other channel.

The clinical literature supplies the evaluation discipline and the negative evidence. A Tutorial on Clinical Speech AI Development documents that reported accuracy falls as sample size grows, because models fit cohort and recording artifacts. It also argues that routing predictions through interpretable constructs, such as the hypernasality measure with 0.9 reliability, acts as regularization. Two Mandarin pathological corpora make the test possible, though neither annotates tone. CDSD: Chinese Dysarthria Speech Database has 44 dysarthric speakers and focuses on speech recognition. DysArinVox has dysphonia and dysarthria patients plus in-setting controls, laryngoscopy, and GRBAS (a standard perceptual voice-quality rating scale) and MOS ratings.

Research Program

The pipeline has three stages: segment the speech, model each contour as a smooth curve, and fit a per-speaker decomposition.

Segmentation. We force-align read speech to its known script. For dysarthric speech, the aligner is a recognizer pre-trained on WenetSpeech and fine-tuned on CDSD, following CDSD's best configuration. We extract log-F0 contours at 20 normalized points per syllable, matching Tian et al.

Curve modeling. We represent each contour by its coordinates on functional principal components fitted to healthy speech from Aishell-1, STCMDS, aidatatang_200zh and MAGICDATA.

Per-speaker scale. For each speaker we compute two anchors from that speaker's own tokens. The first is the mean curve of lexical second tones before a third tone. The second is the mean curve of non-final third tones before non-third tones; these have the same position and the same following-context class as sandhi syllables but no sandhi. Each first-syllable token of a within-word third-tone pair is projected onto the line between the anchors. This gives a position $s$, where 0 means "like this speaker's unaltered third tone" and 1 means "like this speaker's lexical second tone."

Decomposition. We model sandhi positions as

$$p(s) = \pi\,\mathcal{N}(s;\mu,\sigma_1^2) + (1-\pi)\,\mathcal{N}(s;0,\sigma_0^2),$$

where $\pi$ is the application rate, $\mu$ is the realized relative rise when applied (healthy speakers should show $\mu<1$, reflecting incomplete neutralization), and $\sigma_0,\sigma_1$ are the spreads. The scale length $L$ is the distance between the two anchors, compared with age- and sex-matched healthy norms. We use the Tian et al. demographic findings to choose the matching variables. Because pathological speakers contribute few tokens, especially in DysArinVox at about 324 words per person, we fit a hierarchical Bayesian version. It pools across speakers within a group and reports posterior intervals, so a speaker with eight tokens is not over-interpreted.

Handling contexts where sandhi is optional. Sandhi application depends on phrasing and speech rate and cannot be predicted from text alone, as Improved Contextualized Speech Representations for Tonal Analysis notes. The primary analysis therefore uses within-word pairs, where application is near-obligatory. We estimate healthy $\pi$ for these contexts from the large corpora. Cross-boundary pairs are a secondary analysis.

Verification. On a stratified subset of pathological tokens, annotators hand-correct pitch tracks. We report voicing coverage and octave-error rates per speaker. Speakers whose tracking fails beyond a preset threshold are excluded rather than scored. Pitch tracking is the obvious weak point in breathy and tremulous voices, so this step is not optional.

Baselines

Naive scalar reference. This is the per-speaker log-range of the sandhi syllable's rise as defined by Mandarin Tone Sandhi Realization, z-scored against healthy norms. It is the simplest measure a clinic could compute today. Its results are reported for healthy group differences (p < 0.01), not for clinical prediction. It uses exactly the same tokens and alignments as our measure. Beating it means a higher correlation with clinical ratings and higher stability under channel perturbation on identical tokens.

Strongest learned representation. This is the frame-level cross-entropy fine-tuned wav2vec 2.0 from Improved Contextualized Speech Representations for Tonal Analysis. It reports 96.4% frame tone accuracy and 87.7% clustering accuracy on Aishell-1, against 44.4% for scalar F0 features. We retrain it on Aishell-1 with the published recipe and use it two ways. First, we project sandhi tokens onto a speaker-specific axis between lexical second and third tones in its embedding space; this gives the strongest competitor version of our own measure. Second, we train a severity regressor on its pooled embeddings with the same speaker-level cross-validation folds and the same hyperparameter-search budget as our downstream model. There is no reported result on Lacuna for any tone measure, learned or acoustic, on CDSD or DysArinVox. Beating this baseline does not require higher in-sample accuracy. It requires equal-or-better speaker-held-out correlation with ratings, together with clearly smaller sensitivity to recording condition and corpus identity.

Ablations. Three ablations isolate the key ingredients. Replacing the speaker's own anchors with population anchors tests the within-speaker contrast. Fitting a single gradient score instead of the two-part mixture tests whether separating application from realization matters. Using the 20-point raw contour instead of functional components tests the curve representation.

Why not just…?

Why not just use acoustic-to-articulatory inversion to recover sandhi gestures directly? This is the obvious intersection of the three directions, and we do not think it is sound yet. The best available systems, Speaker-Independent Acoustic-to-Articulatory Speech Inversion (Pearson correlation 0.784 on combined English motion-tracking data) and Articulatory-WaveNet (1.08 mm error, correlation 0.82, speaker-dependent), output tongue, lip and jaw trajectories. They output nothing laryngeal, and they were trained only on healthy English. Their outputs on Mandarin dysarthric speech could not be validated against any ground truth in the file, which would fail exactly the analytical-validation stage. For tone, F0 is already the most direct non-invasive readout of the relevant gesture.

Why not just fine-tune a speech foundation model to predict severity end to end? This is the real threat to in-sample numbers, and it is run as a baseline. The tutorial's negative-scaling evidence (A Tutorial on Clinical Speech AI Development) predicts that it will exploit recording differences. Experiments 2 and 3 below measure exactly that. If the end-to-end model matches our measure on held-out speakers and is equally invariant to channel and corpus identity, we would report that the decomposition adds interpretability but not robustness.

Why not just score sandhi with a tone classifier as right or wrong? A categorical classifier treats every healthy speaker's incomplete rise as partly erroneous. It also cannot separate a compressed but planned rise from an absent one. The healthy-corpus results above show the gradient component is systematic. The mixture-versus-single-score ablation tests whether the categorical part carries additional clinical signal.

Why not just use perceptual ratings? Ratings are the clinical outcome we validate against, not a competitor. Human dysarthric transcription error in CDSD (20.45% for experienced and 35.71% for inexperienced transcribers) shows how variable listener judgments are. GRBAS ratings also cannot assign a failed rise to planning versus execution.

Proposed Experiments

Experiment 1: healthy calibration. In the four healthy corpora, the decomposition should recover $\mu<1$ and near-ceiling $\pi$ for within-word pairs. It should also reproduce the published word-frequency and demographic shifts in $\mu$ rather than in $\pi$. This checks that the measure re-expresses known gradient findings.

Experiment 2: channel robustness. We apply codec, noise and reverberation perturbations to healthy and patient audio and compute agreement between the original and perturbed scores for $\pi$, $\mu$, $L$ and every baseline. Where CDSD's ten-hour speakers allow, we add test-retest agreement across recording sessions. We also train a probe to predict corpus of origin (CDSD versus Aishell-1) from each measure. Chance-level prediction from our three numbers, contrasted with near-perfect prediction from embeddings, would directly show the shortcut being removed.

Experiment 3: the decisive dissociation, within DysArinVox. Controls and patients there share a recording setting. We predict that patients with vocal-fold lesions show reduced $L$ and $\mu$. We further predict that $L$ tracks laryngoscopic glottic-closure parameters and GRBAS breathiness, while $\pi$ stays inside the healthy posterior interval. If $\pi$ instead drops in purely peripheral dysphonia, the claimed separation of planning from execution is wrong, and we would report it as such.

Experiment 4: clinical validation within dysarthria. In CDSD and the dysarthric subset of DysArinVox, we correlate the three numbers with available severity or MOS ratings, and with per-speaker transcription error as an intelligibility proxy where ratings are absent. All model selection uses leave-speaker-out folds. We pre-register the comparison against the scalar and embedding baselines.

Token counts per speaker will be reported up front. If CDSD scripts yield too few within-word third-tone pairs, the immediate next step is a short elicitation list for prospective collection, rather than stretching the analysis to optional contexts.