Three’s a Crowd? Effects of a Second Human on Vocal Accommodation with a Voice Assistant

Eran Raveh, Ingo Siegert, Ingmar Steiner, Iona Gessinger, Bernd MöbiusInterspeech 2019

TL;DR

This study investigates how the presence of a second human affects vocal accommodation when interacting with a voice assistant (VA). Researchers found that the presence of a confederate significantly alters participants' speech patterns, leading to less vocal accommodation towards the VA, especially when the interaction with the VA alone occurs first.

What Problem Does This Paper Address?

Voice assistants (VAs) are becoming increasingly integrated into our daily lives, moving beyond individual use to more social contexts where multiple humans interact with the device. While research has explored how humans accommodate their speech to computers and how VAs adapt their output, less is known about human-human-computer interactions. Specifically, this paper addresses the question of whether and to what extent the presence of a second human interlocutor influences a user's vocal accommodation towards a VA. This is crucial for designing more natural and effective human-VA interactions in social settings.

Methods

The researchers employed a two-pronged analytical approach: distributional and temporal.

  • Distributional Analysis: This method compared the overall distributions of phonetic features (fundamental frequency, intensity, and articulation rate) between interactions with a VA alone (solo condition) and interactions with a VA and a human confederate (confederate condition). They used the two-sample Wilcoxon test to detect significant differences in speech patterns. The analysis also considered the order in which the solo and confederate conditions were presented to the participants.
  • Temporal Analysis: This approach examined continuous, time-based changes in vocal accommodation throughout the interaction. They smoothed speech features using LOESS regression to obtain predicted values for each two-second slice of the conversation. A measure of relative change between slices was then used to quantify convergence (moving closer) or divergence (moving apart) between the participant's and the VA's speech. This allowed for a more granular understanding of how accommodation evolves over time and how factors like task, gender, and order influence these dynamics.

Datasets

The study utilized the Voice Assistant Conversation Corpus (VACC) [10]. This corpus contains balanced human-computer (solo condition) and human-human-computer (confederate condition) interactions with a 2nd generation Amazon Echo Dot using Alexa. The corpus includes recordings of 27 German native speakers (14 female) performing two tasks: Calendar (more formal) and Quiz (more informal). Each participant completed both tasks in both solo and confederate conditions, with the order of tasks and conditions counterbalanced. This dataset is publicly available for research.

Key Insights and Ideas

  • The "Third Wheel" Effect: The presence of a second human significantly impacts how users interact with a VA. Users tend to accommodate their speech less to the VA when a confederate is present.
  • Order Matters: The sequence of interactions plays a crucial role. Participants showed significantly more vocal accommodation differences when they first interacted with the VA alone, suggesting a "priming" effect from the initial solo interaction.
  • Social Context Over VA Adaptation: The study suggests that humans are more naturally inclined to accommodate to other humans. When a confederate is present, this natural tendency might overshadow or interfere with the accommodation towards the VA.
  • Ambiguity of Target: With multiple interlocutors, users might struggle to identify a single, consistent target for accommodation, leading to a weakened convergence effect towards the VA.

Key Results

  • Significant Differences: For fundamental frequency (fo) and intensity, a majority of interaction pairs showed significant differences in speech patterns between the solo and confederate conditions. Articulation rate (AR) showed fewer significant differences.
  • Order Effect: When participants interacted with the VA alone first, the percentage of significant differences for fo, intensity, and AR was higher by 12%, 20%, and 3% respectively, compared to when the confederate condition came first.
  • Task Influence: The task performed had a less pronounced effect on accommodation differences compared to the order of conditions, particularly for fo and intensity. For AR, the task did show some differentiating influence.
  • Gender Differences: Female participants showed less convergence towards the VA compared to male participants. The authors speculate this might be related to the VA using a female voice.
  • Solo vs. Confederate: A higher rate of convergence was observed in the solo condition compared to the confederate condition, supporting the idea that the presence of another human reduces accommodation towards the VA.

Limitations and Future Work

  • Confederate's Role: The confederate in the study interacted only with the human speaker, not with Alexa. Future research could explore scenarios where the confederate also directly interacts with the VA.
  • VA Voice: The study used a female-voiced VA. Investigating the impact of a male-voiced VA on gender-based accommodation differences is suggested.
  • Complexity of Social Contexts: The study focused on a specific scenario with one confederate. More complex social dynamics with multiple humans and VAs could be explored.
  • Nature of Accommodation: While the study quantifies vocal accommodation, further research could delve into the qualitative aspects and underlying motivations for these behavioral changes in multi-party interactions.

Summary updated

BibTeX
@inproceedings{raveh2019threes,
  author = {Eran Raveh and Ingo Siegert and Ingmar Steiner and Iona Gessinger and Bernd Möbius},
  title = {{Three’s a Crowd? Effects of a Second Human on Vocal Accommodation with a Voice Assistant}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure