Does Lombard Speech Actually Protect Mandarin Tones? A Resynthesis Test

When people talk in noise, they change how they speak. This is the Lombard effect. In Mandarin, phoneticians have measured these changes carefully and concluded that speakers protect lexical tone, the pitch pattern that distinguishes otherwise identical syllables (mā "mother" from mǎ "horse"). The evidence is that speakers open their mouths wider and lengthen vowels relative to the rest of the utterance, rather than simply getting louder. That conclusion rests entirely on production measurements. Nobody has checked whether a listener, human or machine, actually recovers tones more accurately from Lombard speech. Nobody has checked which of the many simultaneous changes would be responsible if they do. Without that check, "Mandarin Lombard speech safeguards tone" is an untested claim, much like an untested theoretical guarantee.

We propose to test it directly. We take matched plain and Lombard recordings of the same sentences by the same speakers, then build hybrid stimuli with a vocoder that edits pitch, spectrum, and timing independently. Examples include Lombard spectrum with plain-speech pitch, and plain speech with only a global upward pitch shift. We play all of them to tone recognizers and listeners at equalized signal-to-noise ratios. The result is a causal accounting of which acoustic change buys how much tone accuracy. It also tests whether the field's favorite production metric, tone-space dispersion, predicts tone recoverability at all.

Core Claim

Our hypothesis comes from an unresolved tension in the literature. In Cantonese, The effect of lexical frequency and Lombard reflex on tone hyperarticulation found that noise raises mean pitch across all six tones without expanding the spread between tone contours, which it measured as "tone-space dispersion." Rare words, by contrast, do expand that spread for the confusable mid-low tones. That paper explicitly left open whether either change helps listeners. The Mandarin corpus papers, Mandarin Lombard Grid and EMALG, read the higher first formant (F1, a correlate of mouth opening; about 542 to 608 Hz at 80 dB) and the higher vowel-to-utterance ratio as tone-protective. Neither paper ran any recognition test.

Our claim has three parts. First, once the loudness advantage is removed, Lombard speech yields a measurable but modest tone-accuracy gain over plain speech. Second, most of that gain comes from the segmental and timing changes, not from the pitch-contour changes. Third, the pitch component of the gain is reproduced by a simple global pitch shift, which would mean the Lombard pitch change is effort, not contrast enhancement.

Each part can fail independently. If the claim is wrong in the other direction, and Lombard pitch contours carry tone information beyond a global shift, the experiment will show that too.

Research Program

The core resource is the matched-sentence structure of the Mandarin Lombard corpora. Mandarin Lombard Grid (corpus) has 28 speakers reading grid sentences at 30, 55, 70, and 80 dB. EMALG has 34 speakers producing meaningful sentences at 40, 55, and 80 dBA. Lombard Chinese-TIMIT, as used in A comparative study of Grid and Natural sentences effects on Normal-to-Lombard conversion, adds natural continuous sentences. Because the noise was presented to speakers over headphones and the recordings themselves are clean, we can add noise of our choosing afterward.

Every utterance is forced-aligned to syllables. We label surface tones from the known text, applying third-tone sandhi rules and marking neutral tones. We then compute three things per syllable: the pitch trajectory over normalized time, the F1/F2 trajectory, and duration. We also compute the Cantonese paper's tone-space dispersion, adapted to Mandarin's four tones.

Next comes stimulus construction. We use a source-filter vocoder, which separates pitch from the spectral envelope so either can be edited alone. For each speaker, sentence, and noise-level pair, we time-warp the Lombard and plain versions syllable by syllable and build five hybrid conditions. Spectrum-swap keeps the Lombard spectral envelope and timing with the plain pitch contour. Pitch-swap keeps plain spectrum and timing with the Lombard pitch contour. Global-shift is plain speech with its mean pitch moved to the Lombard mean, contour shape unchanged. Duration-only is plain speech with Lombard syllable and vowel durations. Resynthesis-control is each original passed through the vocoder unedited. The resynthesis-control condition absorbs vocoder artifacts, so every comparison is between vocoded signals.

All stimuli are RMS-normalized before mixing with speech-shaped noise and babble. We use a range of fixed signal-to-noise ratios, guided by the noise-type-dependent transition points reported in Mandarin Lombard Flavor Classification. Normalization is essential: without it, Lombard speech wins trivially by being louder. We also report an un-normalized "matched noise level" condition for comparison with prior intelligibility work.

For machine evaluation, we train two models on AISHELL-3 (AISHELL-3), which provides 218 speakers with pinyin transcriptions. The first is a syllable-level tone classifier. The second is a syllable-with-tone recognizer that uses connectionist temporal classification, a training objective that aligns outputs to audio without frame labels; this is the unit choice studied in Comparable Study Of Modeling Units For End-To-End Mandarin Speech Recognition. Training on AISHELL-3 means no recognizer has seen Lombard speech. A third recognizer is a character-level model, because tone errors can be masked in character output. For it we score both character errors and tone errors recovered from the pinyin of the hypothesis.

For human evaluation, native listeners transcribe a stratified subset in pinyin, and we score tone separately from segments.

The statistical analysis follows the warning in Aligning statistical models with inference goals in the neuroscience of language. That paper argues that a model's dependency structure must match the inference goal. Here the goal is to attribute tone accuracy to acoustic components that co-vary within a syllable and are correlated across neighboring syllables. We therefore fit mixed-effects logistic regressions of per-syllable tone correctness on condition, with random effects for speaker and sentence and a term for the preceding syllable's tone. We do not average error rates and compare means.

As a separate question, we ask whether tone-space dispersion predicts per-syllable correctness after controlling for mean pitch and F1. If it does not, the main production metric of the tone-hyperarticulation literature is not measuring tone contrast.

Baselines

The naive reference point is plain speech, vocoded and RMS-normalized, mixed at the same signal-to-noise ratio and scored by the same recognizers and listeners. Every Lombard and hybrid condition is reported as a change in tone error rate relative to it. A Lombard advantage counts only if it exceeds this reference, with a speaker-level bootstrap confidence interval excluding zero.

The second baseline is the most natural existing explanation: Lombard speech is louder and higher-pitched, and nothing more. The global-shift-plus-gain condition implements this explanation exactly, and it uses the same stimuli, recognizers, and listeners as every other condition. The proposal's central claim is beaten if global-shift recovers at least as much tone-accuracy gain as pitch-swap. In that case Lombard pitch contours add nothing beyond effort.

The third baseline is the strongest existing modeling approach to Lombard speech in Mandarin: StarGAN normal-to-Lombard conversion. A comparative study of Grid and Natural sentences effects on Normal-to-Lombard conversion reports an objective intelligibility score (Speech Intelligibility in Bits, SIIB) of 21.963 for the EMALG-trained model and 21.253 for the natural-sentence-trained model. That paper reports no tone accuracy. We will apply its published models, or retrain them on the same data split if they are unavailable, to our plain test utterances. We then score the converted speech exactly like our hybrids. Converted speech that beats plain speech on SIIB but not on tone error rate would show that the intelligibility metric used to evaluate conversion systems misses tone.

No reported result was found on Lacuna for tone error rate, or for any recognition error rate, on Mandarin Lombard Grid, EMALG, or Lombard Chinese-TIMIT. The character error rates in Comparable Study Of Modeling Units For End-To-End Mandarin Speech Recognition (4.89% on read speech with a character attention model) come from proprietary clean data. They are context only, not a comparable target.

The ablation that isolates the key ingredient is the RMS normalization step itself. We run every comparison with and without loudness equalization. This shows how much of any apparent Lombard benefit is simply signal-to-noise ratio.

Why not just…?

Why not just run an off-the-shelf recognizer on EMALG and report error rate by noise level? That is the obvious first study, and we include it as the matched-noise-level condition. On its own, though, it cannot answer the question. Louder speech raises the signal-to-noise ratio, so any gain is confounded with loudness. Character error rate also mixes tone errors with errors the language model corrects or creates. The grid-sentence results in A comparative study of Grid and Natural sentences effects on Normal-to-Lombard conversion show that context availability changes speaker behavior, and therefore recognizer behavior. Even a clean result would say that Lombard speech helps, not why. The resynthesis decomposition is what turns a benchmark number into a test of the phonetic claim.

Why not just compute tone-space dispersion on the Mandarin corpora? That would extend the Cantonese analysis to a new language, but it repeats the original gap: a production measure assumed to track perceptual contrast. We compute dispersion, but treat it as a hypothesis to test against per-syllable tone correctness.

Why not just model Lombard speech as gain plus a pitch shift? This is a serious competitor for the pitch component specifically. The Cantonese finding that noise shifts mean pitch without increasing dispersion predicts that it will win there. We do not argue around it; the global-shift condition tests it directly. The corpus papers' F1 and Alpha ratio findings, including the gender asymmetries in EMALG, suggest it cannot explain the spectrum-swap condition. If spectrum-swap also gains nothing, the "Lombard safeguards tone" claim is falsified, and that outcome is worth publishing.

Why not just train with noise augmentation and ignore real Lombard speech? Investigating the Lombard Effect Influence on End-to-End Audio-Visual Speech Recognition showed in English that plain speech plus added noise misestimates recognition of real Lombard speech in both directions depending on signal-to-noise ratio. Our recognizers are trained with standard noise augmentation, so any residual gap between noisy plain speech and noisy Lombard speech directly measures what augmentation misses for tone.

Decisive Checks

Three checks settle the core claim. The first is the tone error rate gap between Lombard and plain speech at equalized signal-to-noise ratio, across speech-shaped and babble noise, for both recognizers and listeners. If the gap is zero, the safeguarding claim fails at the outset.

The second is the comparison of the spectrum-swap and pitch-swap gains against the global-shift gain. This attributes any benefit to segmental and timing changes versus pitch-contour changes. We expect gains concentrated on the confusable rising and dipping tones (tones 2 and 3), by analogy with the Cantonese mid-low tones.

The third is whether tone-space dispersion predicts per-syllable correctness beyond mean pitch and F1 in the mixed-effects model. We additionally require that the ordering of conditions agrees between machine and human listeners. A machine-only effect would describe the recognizer, not the speech.

A secondary payoff is a tone-preservation audit for normal-to-Lombard conversion. These systems have so far been judged only on SIIB and listener quality ratings. Our hybrid conditions supply a ready-made reference for how much tone accuracy a faithful conversion should deliver.