Articulatory Cooldown: Oversampling Rare Speech Contrasts Along the Hyperarticulation Continuum

When people are misunderstood, they hyperarticulate. They slow down, open the mouth wider and move the lips and tongue harder, so that a word sits farther from the words it could be confused with. Speech classifiers face a related problem when a contrast is rare in training data, such as an infrequent phoneme, a sparsely represented speaker group, or one member of a minimal pair like "peach"/"beach". The standard fix is minority class oversampling: show the rare examples more often. This proposal asks whether the rare examples should be hyperarticulated versions. More precisely, it asks when during training exaggerated examples help and when ordinary casual ones do.

The core idea is a sampling schedule we call articulatory cooldown. Rare-class training draws begin with heavily hyperarticulated tokens of that class and gradually shift toward casual, coarticulated tokens. Coarticulation is the everyday overlap of neighbouring sounds that blurs their boundaries. The intuition is that clear exemplars teach a model where a category lives, while casual tokens near the class boundary teach it where the category ends. Human hyperarticulation is tuned to help listeners. Whether it helps learners, and whether the useful part is the contrast-specific enhancement or the global "slower and louder" boost, is an open and testable question.

Core Claim

We claim three things. First, a measured, per-token hyperarticulation score gives a physically meaningful axis along which to order minority-class oversampling. Second, a hyper-to-casual schedule along this axis recovers more rare-class recall on casual and noisy test speech than duplicating casual tokens, at the same number of minority draws. Third, the benefit comes from contrast-targeted enhancement rather than from global tempo and pitch exaggeration.

The third claim is the risky one, and the design lets it fail cleanly. A geometric argument cuts against the naive version of the idea. Oversampling works by pushing a decision boundary away from a minority cluster that majority gradients have squeezed. Tokens far from the boundary contribute little loss under cross-entropy and should therefore move the boundary little. Hyperarticulated tokens may consequently be good early teachers and poor late ones. That is the prediction the schedule encodes, and the reverse schedule is the test of it.

Grounding in Prior Work

Several hyperarticulation studies motivate treating clarity as a graded, measurable quantity rather than a binary style.

  • Guo and Smiljanic measure coarticulatory resistance on the LUCID corpus. Their metric is the log-Mel spectral distance (dev) between a consonant and the following vowel. They find a graded hierarchy, BABBLE > VOC, L2, READ-CL > NB, READ-CO, and show that imagined barriers in clear read speech raise dev much as real barriers do. We reuse their metric as our per-token score.
  • Roessig et al. show continuous articulatory expansion from background to broad to contrastive focus.
  • Lundmark shows larger lip and jaw acceleration peaks in stressed syllables.

Two lines of perception and production work supply the targeted-versus-global distinction that our key ablation tests.

  • Tokuma and Tokuma found that LUCID clear speech elicited for non-native listeners was identified better in noise by Japanese listeners than clear speech elicited under babble. This suggests that the source of the hyperarticulation matters, not only its amount.
  • Li, Zhou and Xu show that lexical competition locally expands the pitch space of specific Cantonese tone pairs.
  • Garg et al. cut the other way. Mandarin clear speech applied a generic "faster and further" facial boost across all tones, sometimes against the tonal code. This finding is the main reason a global-transform control is mandatory here.

On the oversampling side, the papers constrain the design rather than supply it.

  • Buda et al. found random oversampling to 1:1 dominant for convolutional networks, with no overfitting.
  • Fu et al. found a 0.5:1 ratio better than 1:1 for text classifiers.
  • Li et al. showed that persistent upsampling memorizes rare data. Their COOLDOWN, a high-temperature-then-annealed sampling schedule, reaches +3.1 BLEU on low-resource OPUS-100 languages.

Our method borrows COOLDOWN's two-phase shape. The difference is what gets annealed: COOLDOWN anneals how often rare data appears, while we anneal which rare tokens appear. Global-distribution oversamplers such as OTOS and FreMGP place synthetic samples using generic feature geometry. OTOS deliberately shifts them toward the decision boundary. Our proposal instead orders real and lightly synthesized samples by a physically interpretable clarity axis and tests whether far-from-boundary samples are useful at all.

The intersection itself is nearly empty.

  • Dheram et al. oversample underserved speaker cohorts in production speech recognition but treat the cohorts as acoustic black boxes. They also found no additive gain from combining oversampling with cohort embeddings.
  • Aylett biases unit-selection synthesis toward long, hyperarticulated units. He notes that the method is bounded by how rarely such units appear in the voice database, but does not rebalance that inventory.
  • The multilingual dialogue-correction study documents hyperarticulated repair utterances across nine languages but trains no models on them.

None of these uses hyperarticulation as a sampling axis for rare classes.

Research Program

Testbed. The main testbed is LUCID. It holds the same 40 talkers in casual and clear read speech plus four interactive conditions, which gives natural hyperarticulated and casual tokens of the same words from the same speakers. We build word-level and segment-level classification tasks around confusable contrasts, starting with the minimal pairs Tokuma and Tokuma extracted ("beach"/"peach", "sell"/"shell"). We then artificially impose imbalance so that one side of each contrast is rare, at ratios from 10:1 to 100:1.

Model. The classifier is a small head on frozen self-supervised speech features in the style of vq-wav2vec, plus a fully fine-tuned variant.

Scoring tokens. Every minority token $x$ receives a hyperarticulation score $h(x)$. This is its dev value standardized within speaker and phonetic context, following Guo and Smiljanic. The score lets the natural condition labels be replaced by a continuous measure.

The schedule. At training step $t$, a minority draw picks token $x$ from the minority pool $\mathcal{M}$ with probability

$$p_t(x) = \frac{\exp(\beta_t\, h(x))}{\sum_{x' \in \mathcal{M}} \exp(\beta_t\, h(x'))},$$

where $\beta_t$ is a clarity temperature. It starts at a positive value $\beta_0$, which favours clear tokens, and decays linearly to $0$, which is uniform sampling over casual and clear tokens alike. The overall minority sampling rate is held fixed, so only the composition of the minority draws changes. A negative final $\beta$ is also tested, ending training on the most coarticulated tokens.

Extending the axis synthetically. For sentences read in both styles by the same speaker, we align the casual and clear renditions. We then interpolate, and mildly extrapolate, frame features along the casual-to-clear direction. This yields intermediate and "super-clear" minority samples for classes with few natural clear tokens. The interpolation is the step where targeted and global effects can be separated.

Global control. The global-only version applies to casual tokens the tempo, pitch-range and intensity changes that match clear speech's average statistics, and nothing else.

Targeted version. The targeted version keeps those global statistics fixed. It moves only the feature component that increases distance from the specific competitor class, scaled to the dev gain observed in natural speech.

Transfer check. As a secondary check, we transfer the learned targeted displacement to rare phonemes on TIMIT. TIMIT has no clear-speech recordings, so this tests whether the clarity axis generalizes beyond its source corpus.

Baselines

Naive reference. The naive reference point trains on the imbalanced data with no intervention, using the same model, steps and tuning budget. It shows how much the imposed imbalance costs. We also report the same model trained on the original balanced LUCID data as a ceiling.

Strongest simple method. The strongest simple method is random oversampling of minority tokens with no regard to clarity, at the ratio Buda et al. recommend (1:1) and at the 0.5:1 ratio Fu et al. favour. Both papers found duplication hard to beat, so it is the method any proposal must clear. Its minority pool includes the same clear and casual tokens our method sees, drawn uniformly, so any gain from our method comes from ordering rather than access to extra data.

Strongest existing schedule. The strongest existing schedule is COOLDOWN from Li et al., run on the same pool with its frequency schedule (temperature 5 then 1). The file reports +3.1 BLEU over proportional sampling on OPUS-100 low-resource languages. That number is in a different task. For LUCID, no reported result was found on Lacuna for any oversampling or classification baseline, so all baseline numbers there will be produced by us under identical budgets. Beating COOLDOWN means at least a two-point gain in minority-class recall and G-mean at matched majority accuracy, averaged over held-out-speaker folds. Because articulatory cooldown can be stacked on COOLDOWN, the cleanest comparison is COOLDOWN versus COOLDOWN plus clarity ordering.

Generic synthetic oversampler. The generic synthetic oversampler is OTOS from Yan et al., applied in the same feature space with the same number of synthetic samples. It represents boundary-directed synthesis, the opposite placement philosophy to ours.

Ablations. Several ablations isolate the key ingredient. Fixed clear-only sampling ($\beta$ constant and positive) and fixed casual-only sampling test whether the schedule matters at all. The reverse schedule (casual to clear) tests the order prediction. The global-only synthetic control tests whether the targeted component carries the gain. Source-condition swaps, using L2-elicited tokens versus BABBLE-elicited tokens as the clear pool, test whether listener-directed clarity transfers to learners as Tokuma and Tokuma found it does for human listeners.

Tuning budget. Every arm gets the same grid over its own main knob: ratio, temperature, $\beta_0$, or OTOS regularization. Every arm uses the same number of seeds and the same evaluation split.

Why not just…?

Why not just duplicate casual minority tokens to 1:1, as Buda et al. recommend? This may well be close to optimal for the boundary-shifting role of oversampling, and it is our main baseline rather than a straw man. Buda et al. studied convolutional networks on images. Fu et al. and Li et al. found that persistent duplication overfits in sequence models. Duplication also cannot add variety along the clarity dimension, which is exactly where noisy and casual test speech differs from training speech. If duplication matches our method on casual and noisy test sets, the conclusion is that clarity ordering is unnecessary, and we will report it that way.

Why not just use tempo perturbation or SpecAugment-style masking instead of hyperarticulation? This is the real threat. Garg et al. suggest that much of natural clear speech is a global "faster and further" boost. If so, matching global tempo and pitch statistics should capture most of any gain, and the articulatory story would reduce to a cheaper speed-perturbation schedule. The global-only control is built to settle this question. The targeted version must beat it on the specific competitor contrast, measured as reduced confusion with that competitor rather than overall accuracy. Otherwise we retain the schedule with global transforms and drop the targeted claim. Aylett's finding that duration manipulation alone saturates at about a 17% prominence gain hints that global stretching has limited reach, but that evidence comes from perceived prominence in synthesis, not from classifier learning.

Why not just upweight the minority loss? Li et al. show that loss upweighting and upsampling are equivalent only under full-batch training. Under mini-batch training, upweighting produces high-variance gradients dominated by single outliers. Hyperarticulated tokens are acoustically atypical and may be exactly such outliers. We include an upweighting arm with the same effective minority weight and log per-step gradient norms, so this mechanism is observed directly rather than assumed.

Proposed Experiments

Data splits. All LUCID experiments use held-out-speaker cross-validation. Test sets are NB spontaneous speech (casual, the realistic target), READ-CO, and both of these mixed with multi-talker babble at 0 dB signal-to-noise ratio, matching the condition in Tokuma and Tokuma.

Metrics. We report minority-class recall, G-mean, area under the curve, majority-class accuracy, and the confusion rate with the paired competitor. Human identification of initial /b/ was 68% in babble noise for LUCID tokens (Tokuma and Tokuma). We report model accuracy on the same items as a reference point, not as a target.

Decisive check 1: order. The first check is the order effect. If clear-to-casual beats casual-to-clear and both fixed pools at equal draws, the "teachers early, boundary late" account holds.

Decisive check 2: gradient diagnostics. The second check tests the margin argument directly. We track per-token loss and gradient contribution across training as a function of $h(x)$. If clear tokens dominate useful gradient only early, the mechanism is confirmed. If they never contribute, hyperarticulation is the wrong axis for learners even though it is the right one for listeners. That would be a clean negative result.

Decisive check 3: targeted versus global. The third check is the targeted-versus-global comparison described above.

Decisive check 4: L2-elicited versus babble-elicited clear tokens. The fourth check compares the two clear-token sources as minority material. It asks whether the listener-tailoring that helped Japanese listeners also helps a learning model.

Transfer check on TIMIT. On TIMIT we evaluate rare-phone recall and overall phone error rate with the transferred targeted displacement. Published references are 24.58% label error rate for BLSTM-CTC (Graves-style BLSTM-CTC) and a phone error rate of 11.64 for vq-wav2vec with BERT. The goal on TIMIT is no regression in overall error alongside improved recall on the rarest phones, not a new overall record.