Temporal entrainment in overlapped speech: cross-linguistic study

Marcin Włodarczak, Juraj Šimko, Petra WagnerInterspeech 2012

TL;DR

This paper investigates how speakers synchronize their speech onsets during overlapping conversations across English, French, and German. They found that speakers tend to initiate overlaps not randomly, but rather at specific points within their partner's speech, particularly around vowel onsets, suggesting a universal rhythmic coordination mechanism.

What Problem Does This Paper Address?

Conversations are dynamic, with speakers often talking over each other. While models exist for smooth turn-taking, the temporal coordination during overlapped speech has been less understood. Previous research suggested speakers align their speech rhythmically, but these studies were often limited to a single language or relied on automatic speech segmentation, raising questions about generalizability and accuracy. This paper aims to address this by investigating temporal entrainment in overlapped speech across three different languages, using both automatic and manual segmentation where possible, to see if these rhythmic coordination patterns hold true universally.

Methods

The core method involves analyzing the timing of overlap onsets relative to the speech of the person being overlapped (the "overlappee"). Specifically, they calculated "normalized onset times." For each overlap, they identified the vowel-to-vowel (VTV) interval during which the overlap began. The overlap onset was then normalized by dividing the time from the start of that VTV interval to the overlap onset by the total duration of the VTV interval. This effectively maps overlap onsets onto a scale from 0 (start of the VTV interval) to 1 (end of the VTV interval, i.e., the next vowel onset). They also analyzed syllable boundaries for English and French, mirroring previous work. Statistical tests (Kolmogorov-Smirnov) were used to compare observed distributions against a uniform (random) distribution.

Datasets

  • English: Switchboard corpus (spontaneous, dyadic dialogues)
  • French: CID corpus (spontaneous, dyadic dialogues)
  • German: Kiel corpus of spontaneous speech (manually segmented)

The authors note that for the German corpus, syllable segmentations were not available, so they used vowel-to-vowel intervals as the unit of analysis.

Key Insights and Ideas

  • Universal Rhythmic Coordination: The most striking finding is the consistent pattern of overlap initiation across all three languages. Speakers don't start talking over their partner randomly; they tend to align their speech onsets with specific points in their partner's speech, particularly around vowel onsets.
  • Beyond Syllables: While previous work focused on syllable boundaries, this study demonstrates that similar entrainment patterns are observable even when using vowel-to-vowel intervals, suggesting that the underlying rhythmic anchor might be more perceptually driven (like vowel onsets) than strictly linguistic (like syllable boundaries).
  • Perceptual Centers (p-centers): The authors link their findings to the concept of "p-centers," which are perceptually salient points in speech often coinciding with vowel onsets. This suggests that speakers might be coordinating based on how their partner's speech is perceived, rather than just its structural segmentation.
  • Robustness to Segmentation Method: The fact that the manually segmented German data showed a clear pattern, similar to the automatically segmented English and French data, strengthens the claim that this entrainment is a genuine phenomenon and not an artifact of automatic segmentation.

Key Results

  • Non-Random Overlap Onsets: For all three languages, the distribution of normalized onset times was significantly different from a uniform distribution (p < 0.001 for English and French, p < 0.05 for German). This indicates a non-random tendency in overlap initiation.
  • Peak Alignment: In all corpora, overlap onsets showed a rising trend towards the following vowel boundary, reaching a maximum around 80% of the vowel-to-vowel interval. This means speakers are most likely to start overlapping just before their partner is about to produce the next vowel sound.
  • Cross-Linguistic Similarity: Density functions of normalized onset times were visually very similar across English, French, and German, suggesting a universal aspect of this temporal entrainment.
  • French Anomaly: Interestingly, while the French data showed strong evidence for entrainment based on vowel-to-vowel intervals, analysis using automatic syllable segmentations did not yield a significant deviation from a uniform distribution. This is contrasted with English, where syllable-based analysis did show a pattern. The authors suggest this might be due to the perceptual salience of vowel onsets over syllable boundaries in French.

Limitations and Future Work

  • Syllable Segmentation in German: The lack of manual syllable segmentation for the German corpus limited a direct comparison of syllable-based findings.
  • Automatic Segmentation Concerns: While the German data (manually segmented) supported the findings, the authors acknowledge past concerns about automatic segmentation potentially introducing artifacts.
  • Future Work: The authors suggest their findings can help refine models of temporal alignment in conversations, particularly by providing empirical evidence for rhythmic coordination during overlapped speech, which has been a gap in previous theoretical work. They also highlight the need to further explore the role of perceptual centers and cross-linguistic variations in how these entrainment patterns manifest.

Summary updated

BibTeX
@inproceedings{wodarczak2012temporal,
  author = {Marcin Włodarczak and Juraj Šimko and Petra Wagner},
  title = {{Temporal entrainment in overlapped speech: cross-linguistic study}},
  booktitle = {Interspeech},
  year = {2012},
}

Loading related content...

Figure