TL;DR
This paper investigates how an audience's familiarity with a target accent impacts the perceived success of accent transformation systems. They found that while familiarity generally leads to more favorable evaluations, significant shifts in the relative success of different transformation approaches can occur, underscoring the need for evaluations with target audiences.
What Problem Does This Paper Address?
When we localize speech-enabled software or create synthetic voices, we often aim to adapt the accent to match the target audience. This is crucial for building rapport and improving user experience. However, the "success" of an accent transformation is ultimately judged by the listener, and their perception is influenced by their existing familiarity with the target accent. This paper addresses the question of how this familiarity affects the evaluation of accent transformation systems. Specifically, they want to know if an approach that seems more successful to one audience might be perceived as less successful by another, and to what extent this familiarity influences quantitative measures like Mean Opinion Scores (MOS).
Methods
The authors developed an accent transformation system that modifies both pronunciation and fundamental frequency (Fo) contours.
- Pronunciation Transformation: This was achieved by shifting formants (key acoustic features of vowels) within the F1/F2 space. They preserved the minimum and maximum formant frequencies for each speaker and adjusted higher formants, especially for rhotic phones. The process involved modeling the spectral envelope and then shifting formants based on target values derived from phonetic descriptions.
- Prosodic Transformation: Fundamental frequency contours were adapted by imposing natural contours from similar phrases in the target accent. This was done using a two-tiered Discrete Cosine Transform (DCT) approach, with adjustments for phrase and syllable levels. Dynamic time warping was used to normalize syllable durations, reducing dependencies on phone durations or speech rate.
They then conducted perceptual studies comparing two transformation approaches: one with pronunciation modification only, and another with both pronunciation and Fo modifications.
Datasets
- Source Speech: Utterances from the "North Wind" story, read by male and female speakers from the Keele speech corpus, were used as the source material.
- Target Accents: The target accents included Australian, New Zealand, Irish (specifically northern Irish), and Welsh.
- Prosodic Target Contours: Fundamental frequency contours were taken from similar utterances in the IViE corpus (for British accents) and additional recordings were made for Australian and New Zealand accents.
The study used a novel experimental setup where transformed utterances were presented to two distinct audiences.
Key Insights and Ideas
- Familiarity Matters: The core idea is that listener familiarity with an accent significantly impacts how they perceive accent transformation. This is because unfamiliar listeners rely on a smaller set of salient acoustic features, while familiar listeners can discern finer details.
- Relative Success Can Flip: A crucial finding is that changes in audience familiarity can alter the relative success of different accent transformation approaches. An approach that is significantly better for one audience might not be perceived as better, or could even be perceived as worse, by another audience with different familiarity levels.
- Quantifying MOS Shifts: The study demonstrates that familiarity can be quantified in terms of MOS scores. A shift to a more familiar accent generally leads to a drop in MOS, suggesting a more favorable perception of the original, untransformed speech.
Key Results
- MOS Differences: For both audiences, the original RP (Received Pronunciation) utterances were rated significantly more "RP-like" than any of the transformed versions.
- Familiarity Impact on MOS: Across the study, unfamiliar accents generally resulted in MOS scores that were 0.2 to 0.5 higher than familiar accents. The average increase in MOS for unfamiliar accents was 0.35 (with a standard deviation of 0.16).
- Fo Transformation Effectiveness: The inclusion of Fo modifications consistently improved the perceived success of accent transformation for the New Zealand audience. For the English audience, Fo modifications led to statistically significant improvements for New Zealand and Irish accents, but not for Australian and Welsh accents.
- Relative Success Shifts: The paper highlights specific examples where the relative success of accent transformations changed between audiences. For instance, the Australian accent was perceived as significantly more successful than Irish for the English audience, but less successful for the New Zealand audience. Similarly, the perceived benefit of Fo modifications varied between audiences for certain target accents.
Limitations and Future Work
- Audience Assumptions: The study assumes that the English audience is more familiar with Welsh and Irish accents, and the New Zealand audience is more familiar with Australian and New Zealand accents. While reasonable, these are assumptions.
- Generalizability: The authors acknowledge that conclusions drawn from evaluations with a specific audience cannot be automatically generalized to all audiences.
- Future Work: The paper suggests that while a rough estimation of MOS scores for familiar audiences can be made based on unfamiliar ones, there is no substitute for conducting surveys with the actual target population when comparing transformation approaches or evaluating systems for specific audiences. Further research could explore the specific acoustic features that drive these perception differences across audiences.
Summary updated