TL;DR
While standard French and Swiss French are linguistically similar, the "Swissness" of the accent lies heavily in its rhythm and melody—its prosody. This paper demonstrates that by combining regional pronunciation adaptation with original human timing and intonation, synthetic speech can become indistinguishable from real Swiss speakers in terms of perceived accent.
What Problem Does This Paper Address?
In the world of Text-to-Speech (TTS), making a computer sound like it’s from a specific region is a notorious challenge. For French, the differences between the "standard" Parisian accent and various Swiss accents are subtle at the level of individual sounds (phonemes), but glaringly obvious to native listeners.
Previous research suggested that simply changing how words are pronounced isn't enough to capture a strong regional identity. The authors hypothesize that the "secret sauce" of the Swiss accent is prosody: the patterns of stress, intonation, and duration. They set out to determine if "injecting" real human prosody into synthetic, regionally-adapted speech can bridge the gap between a robotic "standard" voice and a convincing local one.
Methods
The researchers employed a multi-step approach using HMM-based (Hidden Markov Model) speech synthesis:
- Model Adaptation: They started with average French TTS models and used speaker adaptation techniques to create "average regional" models for five specific locations.
- Prosody Supplanting: Instead of letting the computer guess the rhythm, they took the prosodic parameters from original Swiss recordings and "supplanted" (replaced) the synthetic parameters with them.
- Four Experimental Conditions: To isolate what matters most, they created four versions of the same sentences:
- Adapted: Only the regional pronunciation was changed.
- Adapted + Duration: Regional pronunciation plus the original speaker's timing.
- Adapted + Duration + Intonation: Regional pronunciation plus original timing and pitch patterns.
- Vocoded: The original human recording passed through a vocoder (to ensure the "robotic" audio quality was consistent across all tests).
Datasets
The study utilized a specialized dataset of 12 speakers from five distinct locations:
- Switzerland: Geneva (3 speakers), Martigny (1), Neuchâtel (2), and Nyon (4).
- France: Paris (2 speakers) served as the "standard" control group.
For each speaker, 20 sentences were used to adapt the TTS models, with one sentence held back for the subjective evaluation.
Key Insights and Ideas
- Prosody is the Primary Driver: The most striking insight is that pronunciation adaptation alone is insufficient. Listeners perceived a significant jump in "Swissness" only when the timing and pitch were added.
- The "Indistinguishable" Threshold: Once both duration and intonation were added to the adapted synthetic speech, listeners could no longer statistically distinguish the synthetic version from the vocoded original speech in terms of accent degree.
- Rhythm vs. Pitch: Interestingly, the study found no significant difference in accent perception between adding just duration and adding both duration and intonation. This suggests that the unique "slow" rhythm often attributed to Swiss speakers might be more critical for accent identification than the melodic pitch changes.
Key Results
The researchers conducted a subjective evaluation with 19 native French listeners, who rated the "degree of accent" on a scale of 1 (not accented) to 5 (strongly accented).
- Closing the Gap: Combining regional adaptation with original prosody reduced the "perceptual distance" between standard French and the Swiss accent by 41%. This is a marked improvement over the 29% reduction achieved in previous work that didn't use regional pronunciation adaptation.
- Statistical Parity: A Wilcoxon signed-rank test confirmed that there was no significant difference between the scores of the "Duration + Intonation" synthetic versions and the "Vocoded" original recordings.
- Baseline Comparison: The average output with no adaptation scored a 1.42, while the most successful adapted versions reached scores comparable to the original speakers (varying by region).
Limitations and Future Work
While the results are promising, the authors acknowledge a few constraints:
- Small Sample Size: The study relied on only 12 speakers across five locations. Some locations, like Martigny, had only a single speaker, which makes it hard to generalize "average" regional traits.
- Manual Prosody: The study used original human prosody to prove the point. The next big challenge—and the authors' suggested future direction—is to build models that can automatically generate this complex Swiss prosody from text, rather than relying on a human "template."
- Smoothing Effects: The process of averaging multiple speakers for adaptation tends to "smooth out" the very regional quirks they are trying to capture, a common hurdle in statistical parametric synthesis.
Summary updated