Speech dominoes and phonetic convergence

Gérard Bailly, Amélie LelongInterspeech 2010

TL;DR

This paper introduces "speech dominoes," a novel game-based paradigm for collecting rich phonetic data during spontaneous conversation. They demonstrate that this method can reveal subtle phonetic convergence between speakers, particularly for certain vowels and in same-sex dyads, suggesting that adaptation is not always immediate and can be phoneme-specific.

What Problem Does This Paper Address?

The field of speech communication has long been interested in how speakers adapt their speech during conversation, a phenomenon known as phonetic convergence. While theories like the Communication Accommodation Theory (CAT) suggest this adaptation occurs to facilitate communication and build social rapport, empirical evidence for phonetic convergence has been mixed. Previous studies often yielded contradictory results, with some showing little to no adaptation in phonetic variables like loudness, speech rate, or fundamental frequency. This ambiguity stems from two main challenges: the difficulty in collecting sufficient, naturalistic phonetic data that isolates the factors influencing adaptation, and the lack of robust automatic methods to accurately quantify convergence. This paper aims to address these challenges by proposing a new experimental setup and an automated analysis technique.

Methods

The core innovation is the "speech dominoes" game, adapted from a children's language learning activity. In this game, participants select and pronounce dissyllabic words that rhyme with the previously uttered word. This setup encourages active engagement and requires speakers to pay attention to their partner's utterances, creating a naturalistic interactive environment. The researchers collected speech data under various conditions: pre-test (speakers alone), and interactive (speakers playing the game).

For analysis, they employed automatic speech recognition techniques. Specifically, they trained phone-sized, context-independent Hidden Markov Models (HMMs) on the pre-test data. These HMMs were then used to recognize both the pre-test and interactive speech data. Convergence was quantified by comparing the recognition scores of a speaker's own HMMs versus their interlocutor's HMMs. A joint decrease in self-recognition scores and an increase in other-speaker recognition scores were interpreted as evidence of phonetic convergence. This approach, inspired by prior work, leverages MFCC features for spectral analysis, which has been shown to provide robust characterization of convergence.

Datasets

The paper introduces a newly collected dataset generated through the "speech dominoes" game. This dataset consists of several hundred syllables, specifically focusing on eight peripheral oral vowels, collected from speakers in different conditions (pre-test and interactive). The game was designed to elicit approximately 40 exemplars of each target vowel per speaker. The authors do not explicitly name a pre-existing benchmark dataset but rather use the collected data to train speaker-specific HMMs for analysis.

Key Insights and Ideas

  • Game-based data collection is effective: The "speech dominoes" game provides a novel and engaging way to collect rich phonetic data in a controlled yet naturalistic interactive setting, overcoming limitations of previous experimental paradigms.
  • Phonetic convergence is not immediate or universal: The study suggests that convergence of phonetic representations, particularly for vocalic targets, is not always immediate and can be subtle, especially for common words.
  • Convergence is phoneme-specific: The degree of convergence varies significantly across different vowels, indicating that adaptation strategies might be tied to specific phonetic features or articulatory maneuvers.
  • Dyad characteristics matter: The study observed stronger convergence in same-sex dyads, and even noted differences based on relationship status (e.g., a young couple showed different adaptation patterns).
  • Prosody is less affected: Unlike phonetic variables, prosodic features like fundamental frequency and durations showed less significant changes during interaction, suggesting a focus on segmental rather than suprasegmental adaptation.

Key Results

  • The study found occurrences of strong convergence, with some speakers adapting significantly to their interlocutors, while others remained more conservative.
  • In some cases, full convergence was observed for specific vowels, where a speaker's realization became indistinguishable from their partner's.
  • The analysis of vocalic targets revealed that convergence is not systematic across all vowels. For instance, some vowels showed significant minimization of spectral distance, while others did not.
  • Average convergence rates were generally higher for pairs of the same sex.
  • While durations of vocalic nuclei of initial syllables showed some significant changes, prosody overall was relatively unaffected by the interaction.

Limitations and Future Work

The authors acknowledge that their analysis is based on relative distances and that the whole structure of the vocalic space should be considered. They also note that the study focused on peripheral oral vowels and that further investigation into other vowel types and contexts is warranted. Future work includes using this paradigm to study more complex conversational situations, selecting subjects with strong adaptation abilities, and using this data to train speech synthesis engines that can implement interlocutor-aware adaptation strategies for more natural human-virtual agent interaction. The authors also suggest that testing on a larger database is needed to confirm whether target-specific strategies are governed by compact acoustic regions or consistent articulatory maneuvers, and whether face-to-face interaction enhances these.

Summary updated

BibTeX
@inproceedings{bailly2010speech,
  author = {Gérard Bailly and Amélie Lelong},
  title = {{Speech dominoes and phonetic convergence}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure