Pitch similarity in the vicinity of backchannels

Mattias Heldner, Jens Edlund, Julia HirschbergInterspeech 2010

TL;DR

This paper investigates how pitch similarity between speakers influences the perception of backchannels (short feedback utterances like "uh-huh" or "mm-hmm"). They found that backchannels are significantly more similar in pitch to the immediately preceding utterance than other types of turn transitions, suggesting this similarity is a key mechanism for making them unobtrusive.

What Problem Does This Paper Address?

Understanding and generating natural human-like dialogue is a major goal for spoken dialogue systems. A crucial aspect of this is how speakers manage conversational turns and provide feedback. Backchannels, in particular, are short utterances that signal understanding and encourage the speaker to continue, but they must be produced in a way that doesn't disrupt the ongoing conversation. While previous research has noted differences in backchannel duration, loudness, and prosody (like pitch contour), this paper delves deeper into inter-speaker pitch relationships to understand how backchannels are made "unobtrusive." The core question is: how does the pitch of a backchannel relate to the pitch of the speaker it's responding to, and does this relationship differ from other conversational transitions?

Methods

The researchers analyzed spontaneous task-oriented dialogues from the Columbia Games Corpus. They focused on transitions between speakers, specifically comparing backchannels (BC) with smooth switches (S) and pause interruptions (PI), which were grouped as S+PI. They also compared backchannels to other affirmative cue words (AFFCUE) that had similar vocabulary but different discourse functions.

The key innovation here is the focus on inter-speaker pitch similarity. They measured pitch distance in several ways:

  • RAWREL: The raw pitch distance between the end of the preceding utterance and the beginning of the current one.
  • MEANREL & ZREL: Normalized versions of RAWREL, accounting for speaker-specific pitch ranges.
  • OWN: The distance between a speaker's current utterance pitch and their own average pitch over the session (an intra-speaker measure for comparison).
  • OTHER: The distance between the current speaker's pitch and the previous speaker's overall average pitch (a hybrid measure).

They used Praat software to extract fundamental frequency (F0) data and a script called Prosogram to stylize these F0 contours, focusing on perceptually relevant points like syllable nuclei. Statistical analyses, including ANOVAs and Tukey HSD post-hoc tests, were used to determine the significance of observed differences.

Datasets

The study utilized the Columbia Games Corpus. This corpus consists of spontaneous, task-oriented dialogues recorded from native speakers of Standard American English using close-talking microphones. The dialogues were manually transcribed and annotated for phenomena like single affirmative cue words and turn exchanges. The corpus has a total duration of 9 hours and 8 minutes, with individual recording sessions averaging 45 minutes.

Key Insights and Ideas

  • Pitch Similarity as an "Unobtrusiveness" Mechanism: The most striking finding is that backchannels are significantly more similar in pitch to the immediately preceding utterance than other types of turn transitions. This suggests that matching the interlocutor's pitch is a deliberate strategy to make backchannels blend in and avoid disrupting the flow of conversation.
  • Dynamic Inter-Speaker Relations are Powerful: The study demonstrates that simple, dynamic measures of inter-speaker pitch similarity can capture distinctions that more complex, intra-speaker models (like comparing to one's own average pitch) also reveal. This highlights the importance of looking at the interaction between speakers, not just individual behavior.
  • Backchannels are "Higher" in Multiple Ways: Backchannels tend to have higher pitch than a speaker's own average (OWN analysis) and are also more similar to the preceding utterance (RAWREL analysis). This suggests that speakers might be raising their pitch to match a preceding utterance that is already higher than their own average, further emphasizing the adaptive nature of backchannels.
  • Implications for Dialogue Systems: Understanding these inter-speaker pitch dynamics can directly inform the design of more natural spoken dialogue systems. Systems could potentially use this information to decide when to speak, when to remain silent, and how to produce backchannels that are perceived as less intrusive.

Key Results

  • Backchannels are closer in pitch to the preceding utterance: Backchannels showed a mean pitch distance of 0.7 ST from the preceding utterance (RAWREL), compared to 1.8 ST for smooth switches/pause interruptions (S+PI) and -1.7 ST for utterances following a backchannel (X2). All these differences were statistically significant.
  • Backchannels are higher than speaker's own mean: In the OWN analysis, backchannels were significantly higher in pitch (mean 1.7 ST above speaker's mean) than both S+PI (mean 0.7 ST) and X2 (mean 0.2 ST).
  • Backchannels are more similar to preceding utterance than other affirmative cues: Backchannels were 1.3 ST closer to the preceding utterance than other affirmative cue words (AFFCUE), a significant difference.

Limitations and Future Work

  • Limitations: The authors acknowledge that while the statistical differences are significant and perceptually relevant, further work is needed to quantify the exact perceptual impact. The study focused primarily on pitch; incorporating other prosodic features like loudness and intonation could provide a more complete picture.
  • Future Work: The authors suggest tuning the analysis parameters, testing on materials with greater variability (e.g., different noise levels, engagement), and incorporating other inter-speaker features (like loudness) into a dialogue act classification task. They also propose exploring how these findings can be directly applied to improve spoken dialogue system design, particularly in areas like turn-taking and backchannel production.

Summary updated

BibTeX
@inproceedings{heldner2010pitch,
  author = {Mattias Heldner and Jens Edlund and Julia Hirschberg},
  title = {{Pitch similarity in the vicinity of backchannels}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure