TL;DR
This paper investigates whether humans unconsciously adapt their speech (intonation and pronunciation) when interacting with a simulated dialogue system. Using a Wizard-of-Oz setup, they found that users do indeed converge their speech patterns to the system, particularly in question intonation, though individual variation remains significant. This has implications for designing more natural and effective human-computer interactions, especially in educational contexts.
What Problem Does This Paper Address?
The increasing prevalence of spoken dialogue systems (SDSs) in everyday life raises a fundamental question: do humans treat computers as social actors and, consequently, exhibit phonetic accommodation – the unconscious adjustment of speech patterns to match an interlocutor? This phenomenon is well-documented in human-human interaction, often serving as a subtle signal of social connection. However, it's unclear if this behavior extends to human-computer interaction (HCI), especially since computers are not typically perceived as having social intentions. Understanding this could be crucial for improving the naturalness of SDSs and for applications like computer-assisted language learning (CALL), where users might benefit from adapting to a system's pronunciation or prosody. The paper specifically explores accommodation in two key areas: the intonation of wh-questions and the pronunciation of specific German allophonic contrasts.
Methods
The researchers employed a Wizard-of-Oz (WOz) experimental paradigm. In this setup, participants believe they are interacting with an autonomous SDS named Mirabella. However, the system's responses are actually controlled in real-time by a human experimenter (the "wizard"). This allows for the simulation of an intelligent system while maintaining control over its speech output.
The study involved two main types of tasks:
- Intonation Tasks: Participants were engaged in question-and-answer exchanges. Mirabella was programmed to produce questions with specific intonation patterns (falling or rising). The researchers then analyzed the participants' intonation contours for wh-questions to see if they adapted to Mirabella's patterns.
- Segmental Tasks: Participants were asked to describe objects on a map or name pictures. Mirabella was designed to use specific, less preferred pronunciation variants of certain German sounds. The researchers then analyzed whether participants shifted their own pronunciation towards these less preferred variants when interacting with Mirabella.
Perceptual classification by trained phoneticians was used for intonation, while acoustic measurements (formant frequencies) and perceptual categorization were used for segmental analysis. Statistical analysis involved generalized linear mixed-effects models (GLMMs) to account for individual user variability.
Datasets
The study did not introduce new datasets but rather collected data from 20 native German speakers participating in the described WOz experiment. The data consists of recorded speech from these interactions.
Key Insights and Ideas
- Humans do phonetically accommodate to computers: The study provides evidence that phonetic convergence, a hallmark of human-human interaction, also occurs in human-computer interaction, even when the computer is a simulated SDS.
- Intonation is more susceptible to accommodation than segmental features: Users showed a stronger tendency to adapt their question intonation patterns to match the system compared to their pronunciation of specific allophonic contrasts.
- Individual variation is key: While group-level trends show convergence, the degree and even direction (convergence, maintenance, or divergence) of accommodation varied significantly across individuals and features. This highlights the complexity of accommodation and suggests that a one-size-fits-all approach to designing adaptive systems might not be effective.
- The "Wizard-of-Oz" method is effective for studying subtle interaction phenomena: This approach allows researchers to control system behavior in a way that mimics a real SDS, enabling the investigation of phenomena like phonetic accommodation without the need for fully developed AI.
Key Results
- Intonation:
- Users predominantly used falling intonation for wh-questions when the system did.
- When Mirabella produced questions with rising intonation, the number of user questions with rising intonation increased significantly (by 52% from round 1 to round 2 of task 3).
- This convergence was stronger for the overall rising contour than for the specific placement of the nuclear pitch accent.
- Segments:
- For the [Iç] vs. [ik] contrast, the use of the dispreferred variant by users increased by 27% during the map task compared to the baseline.
- For the [ε:] vs. [e:] contrast, no significant convergence was found at the group level, suggesting a general maintenance of preferred variants. However, individual users did show convergence or divergence.
- Individual Behavior:
- Almost every user showed substantial convergence to the system for at least a subset of the examined features.
- Divergence was observed in a few individual cases, particularly for the allophonic contrasts.
- User accommodation patterns were not consistent across all features; a user might converge on one feature but maintain or diverge on another.
Limitations and Future Work
- Limitations:
- The study focused on native German speakers. The findings might differ for non-native speakers, especially in a CALL context.
- The study used natural speech recordings for Mirabella. Future work could explore whether synthetic speech elicits similar accommodation effects.
- The analysis of segmental features primarily relied on perceptual categorization and acoustic measurements of formant frequencies. A more direct categorical analysis of the [ε:] vs. [e:] contrast might provide further insights.
- Future Work:
- Extend the study to non-native speakers of German.
- Utilize synthetic speech for Mirabella's utterances to investigate the impact of speech synthesis quality on accommodation.
- Further investigate the qualitative differences in accommodation across various features and individual users.
Summary updated