TL;DR
This paper investigates which prosodic features—timing or pitch—are more critical for native Japanese speakers judging the naturalness of non-native (English-accented) Japanese speech. They found that timing errors significantly degrade naturalness more than pitch errors, suggesting timing is a primary factor for L2 speech assessment.
What Problem Does This Paper Address?
The naturalness of non-native speech is a complex phenomenon influenced by various linguistic features. While it's generally accepted that prosodic errors (like those in rhythm, intonation, and stress) have a greater impact on naturalness than segmental (pronunciation) errors, the relative importance of individual prosodic features remains less clear. Specifically, for Japanese, a pitch-accent language, the interplay between timing and pitch in perceived naturalness needs further exploration. Previous studies often relied on natural speech, making it difficult to precisely control error types and amounts. This paper aims to address this by using synthesized speech to isolate and manipulate timing and pitch errors, allowing for a more controlled investigation into their impact on native listeners' naturalness judgments.
Methods
The researchers employed a speech morphing system called STRAIGHT to create synthesized Japanese utterances. They started with two sets of speech: one from a native Japanese speaker and another from a bilingual speaker mimicking typical English-accented Japanese. The key innovation here is the ability to independently manipulate the pitch and timing contours of the native speech by morphing them with the corresponding contours from the accented speech. This allowed for the creation of stimuli with varying degrees of pitch and timing errors (0%, 50%, 100%) while keeping the segmental information constant. Native Japanese listeners then rated the naturalness of these synthesized utterances on a 7-point Likert scale.
Datasets
The study did not use pre-existing datasets for training or evaluation. Instead, they created their own experimental stimuli. The core "dataset" consists of:
- Original native Japanese speech samples: Used as a baseline for correct prosody.
- English-accented Japanese speech samples: Created by a bilingual speaker mimicking typical L2 Japanese pronunciation errors.
- Synthesized stimuli: Generated by morphing the native and accented speech samples with controlled proportions of pitch and timing modifications.
Key Insights and Ideas
- Timing is King (for Naturalness): The most striking finding is that timing errors have a significantly greater negative impact on the perceived naturalness of L2 Japanese speech than pitch errors. Even when pitch was completely incorrect (100% morphing), the speech was rated more natural than when timing was incorrect (even at 50% morphing).
- Pitch is Less Critical: While pitch is important for Japanese pitch-accent, its deviation from the norm appears to be more tolerated by native listeners than timing deviations. The addition of pitch errors to already timing-impaired stimuli did not further decrease naturalness significantly.
- Controlled Synthesis is Powerful: Using STRAIGHT to precisely control prosodic features allows for a more nuanced understanding of their individual contributions to naturalness, overcoming limitations of studies using only natural speech.
- Implications for L2 Teaching: The findings suggest that focusing on timing correction in L2 Japanese pronunciation training might yield more significant improvements in perceived naturalness than solely focusing on pitch.
Key Results
The study presented results in terms of z-score normalized naturalness ratings. Key findings include:
- The perfectly natural stimulus (p000t000) received the highest score.
- The stimulus with 100% incorrect prosody (p100t100) received the lowest score.
- Crucially, the stimulus with correct pitch but 100% incorrect timing (p000t100) was rated significantly lower than the stimulus with correct timing but 100% incorrect pitch (p100t000). This directly supports the idea that timing is more important.
- When comparing stimuli with 50% morphing, the one with correct timing (p000t050) was rated higher than the one with correct pitch (p050t000), though this difference was less pronounced than at 100% morphing.
Limitations and Future Work
- Stimuli Creation: The study acknowledges the difficulty in creating perfectly "natural" erroneous speech. The synthesized stimuli, while controlled, might not perfectly replicate all nuances of human L2 speech. The researcher who created the stimuli was a phonologist and experienced teacher, which is a strength, but the "absolute beginner's pronunciation" was mimicked.
- Generalizability: The study focused on English-accented Japanese. While the authors suggest timing's importance might extend to other languages, this needs further empirical validation.
- Future Work: The authors suggest that the significance of timing in naturalness judgment should be tested in other languages. They also propose that timing correctness could serve as a valuable assessment scale for L2 speaker proficiency. Further research could explore the interaction of timing and pitch with other prosodic features like intensity.
Summary updated