Foreign accent matters most when timing is wrong

Chiharu TsurutaniInterspeech 2010

TL;DR

This study reveals that for Japanese learners of English, timing errors are perceived as more detrimental to intelligibility and naturalness than pitch errors by native speakers. This finding challenges the traditional emphasis on pitch in pronunciation assessment and suggests a shift in focus towards temporal accuracy for effective L2 learning.

What Problem Does This Paper Address?

Improving the pronunciation of second language (L2) learners is a key goal in language education, with a particular focus on prosody (pitch, duration, loudness). While pitch accent is crucial in Japanese, and previous research suggested pitch's dominance in L2 speech assessment, the relative importance of different prosodic features for intelligibility and naturalness remains unclear. Specifically, this paper investigates whether pitch errors or timing errors have a greater impact on native Japanese speakers' perception of L2 English learners' speech. Understanding this hierarchy is vital for developing effective pronunciation training and computer-based assessment tools that prioritize the most impactful features for learners.

Methods

The researchers designed an experiment to test the perceptual impact of pitch and timing errors in Japanese utterances produced by English L2 learners. They selected utterances from a corpus of Australian English speakers studying Japanese, specifically choosing examples with:

  1. Incorrect pitch, correct timing (Px)
  2. Correct pitch, incorrect timing (Tx)
  3. Correct pitch, correct timing (O - control)
  4. Incorrect pitch, incorrect timing (X - combined errors)

These stimuli were then presented to 80 native Japanese speakers (university students) who rated the naturalness of each utterance on a 7-point Likert scale. The study also included a recording task where native speakers produced Japanese sentences to establish baseline prosodic patterns, confirming that phrase-final pitch raising and lengthening are common in casual speech.

Datasets

The study utilized a dataset of Japanese utterances produced by Australian English speakers who had studied Japanese for 160 hours. This dataset was curated to specifically include examples exhibiting timing errors, pitch errors, or both, alongside correctly produced utterances. No new benchmark datasets were introduced; rather, existing speech data was carefully selected and categorized.

Key Insights and Ideas

  • Timing Trumps Pitch: The most significant finding is that timing errors, particularly at the word level, have a much greater negative impact on native speakers' perception of naturalness and intelligibility than pitch errors.
  • Pitch Errors Tolerated: Native Japanese listeners demonstrated a notable tolerance for pitch deviations, even when they were not in line with standard Japanese pitch patterns.
  • Phrase-Final Prosody is "Natural": Pitch raising and final syllable lengthening at the phrase level, common in casual Japanese speech, were not perceived as errors by native listeners. This suggests these features are integrated into natural speech behavior and don't necessarily detract from acceptability.
  • Implications for Assessment: The findings strongly suggest that current L2 pronunciation assessment tools, which may overemphasize pitch, should be re-evaluated. Prioritizing timing accuracy in training and assessment could lead to more efficient improvements in L2 intelligibility.

Key Results

The listening task revealed a clear hierarchy of perceived error severity. The average scores (indicating lower acceptability) were highest for utterances with combined timing and pitch errors (X), followed by those with only timing errors (Tx), and then those with only pitch errors (Px). Utterances with correct timing and pitch (O) received the highest scores.

Specifically:

  • A linear regression analysis showed a significant relationship between the number of timing errors and lower performance scores (p<.000).
  • Conversely, the relationship between scores and word pitch errors, or phrase-level errors, was not significant.
  • Multiple t-tests confirmed significant differences in listener judgments between all four error pattern comparisons, underscoring the distinct impact of each error type.

Limitations and Future Work

The authors acknowledge that the stimuli were natural utterances, making it difficult to precisely control the degree of incorrectness compared to synthesized speech. While efforts were made to select speech with similar speech rates and numbers of errors, some variability likely remained.

Future work could involve:

  • Investigating native speakers from different regions (e.g., Tokyo) to confirm the generalizability of findings, although regional differences in perception are not expected to be significant.
  • Exploring the impact of prosodic errors in L2 speech from learners with different L1 backgrounds.
  • Expanding the study to other languages with different prosodic organizations to understand universal vs. language-specific prosodic priorities.
  • Further research into the specific types of timing errors that are most detrimental.

Summary updated

BibTeX
@inproceedings{tsurutani2010foreign,
  author = {Chiharu Tsurutani},
  title = {{Foreign accent matters most when timing is wrong}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure