TL;DR
This paper investigates whether users naturally incorporate "acknowledgments" (like "uh-huh" or "okay") into human-computer interactions, even when the system doesn't explicitly prompt for them. They found that a significant portion of users do use these conversational cues to manage dialogue flow, suggesting that current HCI designs might be unnecessarily suppressing a natural human interaction pattern.
What Problem Does This Paper Address?
Human-computer interaction (HCI), particularly with spoken-language interfaces, often simplifies dialogue to focus on core task completion. This simplification frequently leads to the suppression of "meta-dialogue acts" like acknowledgments and repetitions. These are conversational cues that signal understanding and help manage the flow of conversation, but they don't directly contribute new information. Current systems often discourage their use through rigid turn-taking structures or by cutting off user input (barge-in). The authors question whether this suppression is necessary or if users would naturally employ these human conventions if given the opportunity, and if doing so would be beneficial.
Methods
The researchers employed a "Wizard of Oz" (WOZ) protocol to simulate a spoken-language interface. This allowed them to control the system's responses while giving the impression of a real interface. The core of their method involved designing a task where users had to transcribe information from spoken messages. This task was chosen because it naturally creates a need for users to control the pace of information delivery, a situation where acknowledgments are common in human-human conversation.
The interface presented messages in segments, followed by a pause. Users could either wait for the system to continue (time-out), issue a command (e.g., "go on"), or use an acknowledgment (e.g., "okay"). The system was designed to respond to both commands and acknowledgments by proceeding to the next segment, thus allowing users to choose their preferred method of pace control. The researchers then analyzed user behavior based on whether they used time-outs, commands, acknowledgments, or a combination of these.
Datasets
Not discussed in the paper. The study used a custom-designed task with pre-recorded email messages delivered via a synthesized voice.
Key Insights and Ideas
- Natural Inclination: Users will use acknowledgments in HCI, even without explicit prompting or system support, when the interaction design creates a need for them.
- Pace Control as a Motivator: Tasks requiring users to process and transcribe information are fertile ground for eliciting acknowledgments, as users naturally seek to control the pace of information delivery.
- Current Designs May Be Too Restrictive: The study suggests that current spoken-language interface designs, by suppressing meta-dialogue acts, might be hindering more natural and potentially efficient user interactions.
- "Strange" but Effective: Some users found using acknowledgments to a computer "strange" but were pleased when it worked, indicating a potential for user adaptation and acceptance.
- Politeness as a Fallback: Politeness behaviors (like "please") were observed, especially early in interactions, suggesting users may fall back on human conventions when facing an unfamiliar dialogue situation, rather than necessarily indicating genuine politeness.
Key Results
- Widespread Use: 8 out of 14 subjects (57%) used an acknowledgment or repetition at least once.
- Preference for Acknowledgments: 4 out of 14 subjects (29%) used acknowledgments/repetitions more frequently than commands to control pacing.
- No Exclusive Reliance on Time-outs: No subject relied solely on time-outs; even those who used them predominantly also employed commands or acknowledgments.
- Gender Differences (Tentative): While not statistically significant due to small sample size, the study noted that all male subjects used acknowledgments at least once, whereas 40% of female subjects did. However, a higher percentage of female subjects used acknowledgments in preference to commands.
- User Surprise: Two subjects who used acknowledgments expressed surprise that they worked, and two who didn't use them reported they would have if they had known it was an option.
Limitations and Future Work
- Small Sample Size: The study acknowledges that the small sample size (14 subjects) prevents statistically significant conclusions, particularly regarding gender differences.
- Artificial Interface: The WOZ setup and synthesized voice were deliberately artificial to emphasize the system's limitations and avoid leading users. This might have influenced behavior.
- No Functional Benefit: Users received no direct functional benefit from using acknowledgments in this study; future work could explore the impact of such benefits.
- Future Directions:
- Conducting the study with a larger, gender-balanced group to draw firmer conclusions.
- Comparing the user acceptability and usefulness of dialogue models with and without acknowledgment behavior.
- Investigating the effect of synthesized voice quality on acknowledgment incidence.
- Exploring how politeness and acknowledgment behaviors evolve over longer interactions and with increased user familiarity.
Summary updated