Prosodic parallelism as a cue to repetition and error correction disfluency

Jennifer Cole, Mark Hasegawa-Johnson, Chilin Shih, Heejin Kim, Eun-Kyung Lee, Hsin-Yi Dora Lu, Yoonsook Mo, Taejin YoonDiSS 2005

TL;DR

This paper proposes that the rhythmic and melodic similarity (prosodic parallelism) between the repeated part of a disfluency and its correction acts as a crucial signal for listeners, aiding comprehension. By analyzing spontaneous speech, the researchers found that this parallelism is a common and important feature of repetition and error-correction disfluencies, suggesting it plays a key role in how we process and understand imperfect speech.

What Problem Does This Paper Address?

Conversational speech is peppered with disfluencies – hesitations, repetitions, and corrections – that, surprisingly, rarely impede understanding. While we intuitively process these "imperfections," the specific acoustic cues that help listeners navigate them remain an active area of research. This paper focuses on complex disfluencies involving word repetition and error correction, aiming to identify the acoustic-prosodic features that signal these events to the listener and facilitate comprehension, despite their disruptive nature. The authors hypothesize that a specific prosodic pattern, "parallelism," between the repeated segment (reparandum, REP) and the corrected segment (alteration, ALT) is a key perceptual cue.

Methods

The researchers employed a multi-stage approach:

  1. Transcription and Labeling: A corpus of spontaneous telephone conversations (Switchboard) was meticulously transcribed for disfluencies. Labelers, trained in acoustic phonetics and prosodic transcription (using ToBI conventions), identified and segmented disfluent regions into components like the reparandum (REP), edit phrase (EDT), and alteration (ALT). They used a combination of lexical, syntactic, and prosodic factors for classification.
  2. Prosodic Classification: Complex disfluencies (Repetition and Error Correction) were further categorized into five sub-classes based on the prosodic relationship between the REP and ALT intervals. These sub-classes included:
  3. -same: Highly similar prosodic patterns.
  4. -fp: ALT prosody characteristic of a filled pause.
  5. -ip: Perceived intonational phrase boundary at REP onset, with a pitch reset at ALT onset.
  6. -exaggerated: ALT prosody is a scaled-up version of REP prosody.
  7. -change: ALT prosody differs significantly from REP prosody.
  8. Acoustic Analysis (F0): To quantitatively assess prosodic similarity, the researchers analyzed fundamental frequency (F0) contours. They extracted smoothed F0 contours, normalized them for time, and calculated differences and distances between REP and ALT F0 patterns. This involved measures like trimmed F0 difference and trimmed F0 distance.

Datasets

The primary dataset used was the Switchboard corpus, a collection of spontaneous informal telephone conversations. The authors selected 70 sound files, totaling approximately 60 minutes of speech for detailed transcription and labeling. A subset of these files was used for inter-labeler reliability checks. No new benchmark datasets were introduced.

Key Insights and Ideas

  • Prosodic Parallelism as a Signal: The most striking finding is the prevalence of "prosodic parallelism" – where the pitch contour of the repeated part (REP) closely mirrors that of the corrected part (ALT). This similarity is hypothesized to be a strong perceptual cue for listeners, signaling the presence and nature of the disfluency.
  • "Same" is Most Common: The sub-class labeled "-same," indicating highly similar prosodic patterns between REP and ALT, was the most frequent category for both hesitation-repetition and repair-error correction disfluencies. This suggests that listeners are highly attuned to this rhythmic and melodic consistency.
  • Beyond Simple Repetition: The study moves beyond simply identifying disfluencies to classifying them based on nuanced prosodic relationships. This finer-grained analysis reveals distinct acoustic signatures for different types of disfluencies, such as the "exaggerated" or "ip" (intonational phrase boundary) patterns.
  • Listener-Centric Approach: The research is grounded in the idea that disfluencies are not just speech errors but also communicative acts. The focus on perceptual cues highlights how speakers implicitly guide listeners through these moments of speech production.

Key Results

  • High Inter-Labeler Agreement: The labeling scheme demonstrated good reliability, with an agreement rate of 86.82% for disfluency type and 85.07% for the prosodic sub-classes.
  • Prevalence of Parallelism: The "-same" prosodic sub-class accounted for nearly half of the complex disfluencies analyzed, underscoring the importance of prosodic parallelism.
  • Quantitative F0 Evidence: F0 difference and distance metrics provided quantitative support for the perceptual labeling. For the "-same" sub-class, these measures showed small values, indicating similar F0 contours between REP and ALT. Conversely, other sub-classes like "-exaggerated" and "-ip" showed distinct F0 patterns, confirming the acoustic correlates of the perceptual categories.

Limitations and Future Work

  • Limitations:
    • The study focused on spontaneous telephone speech, which may not generalize perfectly to all speech contexts (e.g., face-to-face interaction).
  • While inter-labeler agreement was good, subjective perception still plays a role in prosodic classification.
  • The "-change" sub-class did not show a strong enough effect to fully support the initial expectation, suggesting further investigation is needed.
  • Future Work:
    • The authors suggest that the frequent occurrence of prosodic parallelism could serve as an important perceptual cue for listeners, aiding in the "online editing" of disfluencies. Future research could explore how listeners utilize these cues in real-time comprehension.
  • Further investigation into the "-change" sub-class is warranted to better understand its prosodic characteristics.
  • Expanding the analysis to other types of disfluencies and different speech corpora could provide a more comprehensive understanding of prosodic cues in speech production.

Summary updated

BibTeX
@inproceedings{cole2005prosodic,
  author = {Jennifer Cole and Mark Hasegawa-Johnson and Chilin Shih and Heejin Kim and Eun-Kyung Lee and Hsin-Yi Dora Lu and Yoonsook Mo and Taejin Yoon},
  title = {{Prosodic parallelism as a cue to repetition and error correction disfluency}},
  booktitle = {DiSS},
  year = {2005},
}
Figure