Variability and stability in collaborative dialogues: turn-taking and filled pauses

Štefan BeňušInterspeech 2009

TL;DR

This paper investigates the intricate relationship between filled pauses (like "um" and "uh") and turn-taking in human conversations. By analyzing spontaneous dialogues, they reveal which acoustic and temporal features of these pauses are stable across speakers and which are influenced by conversational partners, offering insights for more natural dialogue systems.

What Problem Does This Paper Address?

Conversational AI systems often struggle to mimic the natural flow of human dialogue, particularly in managing turn-taking and incorporating subtle cues like filled pauses. Filled pauses, far from being mere speech errors, serve crucial functions in signaling cognitive load, planning, and managing conversational floor. Understanding the variability and stability of these functions, especially how they interact with turn-taking and are influenced by conversational partners, is key to building more human-like and effective dialogue systems. This paper aims to shed light on this by analyzing how filled pauses are used in collaborative task-oriented dialogues, identifying patterns that are robust across speakers versus those that are more dynamic and context-dependent.

Methods

The researchers analyzed a corpus of spontaneous, task-oriented dialogues. They employed quantitative analysis of acoustic and temporal features of filled pauses. Key methods include:

  • Automatic Acoustic Feature Extraction: Using tools like Praat to extract features such as pitch, intensity, and formant values.
  • Prosodic Transcription: Using ToBI conventions to annotate pitch accents and boundary tones.
  • Statistical Analysis: Employing ANOVA and post-hoc Tukey tests to identify significant differences and correlations in acoustic features across speakers and conversational partners.
  • Temporal Analysis: Measuring durations of filled pauses and surrounding silent pauses, and calculating "pause ratios" to understand their relative timing.
  • Categorization of FP Positions: Classifying filled pauses based on their position within turns and conversational chunks (e.g., turn-initial, chunk-final).

Datasets

The study utilized the Columbia Games Corpus. This corpus consists of 12 dyadic spontaneous task-oriented conversations involving 13 speakers of standard American English (7 males, 6 females). Subjects played collaborative games (CARDS and OBJECTS) where they could not see each other, limiting non-verbal cues. The dialogues were recorded, digitized, and downsampled to 16K. The transcripts were orthographically transcribed, and words were aligned to the acoustic signal.

Key Insights and Ideas

  • Filled Pauses as Delimiters: Filled pauses, particularly at the beginning or end of turns and chunks, play a significant role in managing conversational flow and signaling floor-holding or floor-grabbing intentions.
  • Turn-Initial FPs are Salient: Turn-initial filled pauses are particularly important, often serving as "entry devices" for speakers to plan their utterance and for listeners to prepare for incoming information.
  • Intensity is Highly Variable: Acoustic features like intensity are the least stable, showing significant variation both across different speakers and even within the same speaker depending on their conversational partner. This suggests intensity is prone to local entrainment.
  • Temporal Relations are Stable: The temporal relationships between filled pauses and surrounding silent pauses (e.g., the ratio of FP duration to preceding/following silence) are remarkably stable, especially in turn-initial positions. This indicates FPs are integrated into the speech rhythm.
  • "Um" vs. "Uh" Nuances: While "um" and "uh" might have different functional tendencies (e.g., "um" more often followed by silence), the core temporal relationship between the FP and subsequent silence is similar once silence occurs, suggesting a shared underlying mechanism rather than a strict lexical difference.
  • Pitch and Duration Show Moderate Stability: Pitch-related features and plain duration of FPs show less speaker variability than intensity, particularly in turn-initial positions, but are still more variable than temporal relations.
  • Entrainment is Real: Some speakers show significant differences in their FP use (e.g., proportion of turn-initial FPs) depending on their conversational partner, highlighting the dynamic nature of collaborative dialogue.

Key Results

  • FP Distribution: 32.9% of FPs start a turn, and 16.7% are chunk-final, suggesting strong delimitative functions. Turn-final FPs are rare (3.3%), indicating they are not typically used for yielding the floor.
  • Pause Ratio Stability: The "following pause ratio" (FP duration / following silent pause) and "preceding pause ratio" (FP duration / preceding silent pause) were found to be relatively stable features, with minimal between-speaker or interlocutor effects, especially for turn-initial FPs.
  • Intensity Variability: Intensity and raw duration were the most variable acoustic features across speakers.
  • Entrainment Examples: Speaker 103 showed a significant change in the ratio of turn-initial FPs depending on the interlocutor (50% with one, 78% with another). Speaker 103 also exhibited differences in intensity, pitch, and duration based on the interlocutor.
  • Pitch Accent: Turn-initial FPs were significantly more likely to receive a high pitch accent (H) compared to turn-internal FPs, which were more likely to have a low (L) or no accent. This tendency was primarily driven by nasal FPs ("um", "mm").

Limitations and Future Work

  • Limited to American English: The findings are specific to American English, and the authors note that these patterns might not directly translate to other Germanic languages or even British English, warranting further cross-linguistic research.
  • "Um" vs. "Uh" Lexicality: While the study suggests that the functional differences between "um" and "uh" might be reducible to continuous temporal features, the notion of categorical lexical differences is not entirely dismissed and could be a subject for future investigation.
  • Entrainment Complexity: While entrainment was observed, the factors driving it (e.g., specific conversational strategies, personality) are complex and could be explored further.
  • Dialogue System Integration: The authors suggest that understanding these stable and variable features is crucial for designing dialogue systems that can either generalize well (using stable features) or adapt to users (using variable features). Future work could focus on how to practically implement these findings in AI.

Summary updated

BibTeX
@inproceedings{benus2009variability,
  author = {Štefan Beňuš},
  title = {{Variability and stability in collaborative dialogues: turn-taking and filled pauses}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure