Look but Don't Stare: Mutual Gaze Interaction in Social Robots

Yanxia Zhang, Jonas Beskow, Hedvig KjellströmICSR 2017 · DOI

TL;DR

This paper introduces a novel interactive mutual gaze model for social robots, moving beyond simple head movements to simulate nuanced eye contact. Their findings demonstrate that this more sophisticated gaze behavior significantly enhances social connectedness, rapport, and overall user experience in human-robot interactions.

What Problem Does This Paper Address?

As robots become more integrated into our lives for tasks like tutoring, rehabilitation, and assistance, creating natural and engaging human-robot interaction (HRI) is crucial. While previous research has explored using gestures and speech for HRI, non-verbal communication, particularly eye gaze, remains a key area for improvement. Humans are highly attuned to eye contact, using it to signal attention, intention, and regulate social dynamics like intimacy and affiliation. However, existing robotic systems often rely on crude head movements to mimic gaze, which can feel unnatural and fail to convey the subtle social cues embedded in human eye movements. This paper addresses the gap in understanding how implementing mutual gaze – genuine eye-to-eye contact – in social robots can positively impact user perception and interaction quality, moving beyond simply making robots look realistic to making them feel more socially present.

Methods

The authors developed an interactive mutual gaze model implemented on the Furhat social robot head. The system uses a Tobii EyeX Controller to track the user's gaze in real-time. The core of their approach lies in a state transition model that governs the robot's gaze behavior based on the user's gaze status (looking at the robot or away).

Key aspects of their method include:

  • Real-time Gaze Tracking: The EyeX controller captures eye ball positions and gaze at a high frame rate.
  • Gaze Region Definition: A specific facial region on the robot's face is defined as the target for gaze detection.
  • Gaze State Classification: A window-based approach is used to filter out rapid eye movements (saccades) and classify the user's gaze as either "on" the robot or "away."
  • State Transition Model: A deterministic state machine, visualized in Figure 2, dictates how the robot's gaze shifts between looking at the user (FF, FA) and looking away (AF, AA). This model incorporates parameters like the duration of mutual gaze (Tff) and the time before gaze aversion (Taf), informed by human interaction studies.
  • Animated Gaze: Crucially, the robot's gaze is conveyed through a 3D animation projected onto a translucent mask, allowing for subtle, fine-grained eye movements rather than mechanical head turns. This is a significant departure from previous methods that relied solely on head orientation.

Datasets

Not discussed in the paper. The study involved a user study with 24 participants, and data was collected through eye-tracking and questionnaires.

Key Insights and Ideas

  • Subtle Gaze Matters: The paper strongly suggests that nuanced, animated eye movements are more effective for conveying social attention than simple head movements. This aligns with the understanding that human eye movements are subtle and high-frequency.
  • Mutual Gaze Enhances Rapport: The implementation of mutual gaze significantly improved users' perception of rapport with the robot, suggesting that eye contact is a fundamental building block for social connection, even with artificial agents.
  • Reduced Nervousness: In conversational tasks, users reported feeling less nervous when interacting with the robot exhibiting mutual gaze, indicating that a more natural gaze behavior can alleviate social anxiety in HRI.
  • Improved Task Fluency: For cognitively demanding tasks like a quiz game, mutual gaze led to more fluent interactions and faster task completion, implying that effective gaze can streamline cognitive processes during interaction.
  • Attention Allocation: Users allocated more visual attention to the robot in the mutual gaze condition, suggesting that the robot's engaging gaze behavior captured and held user attention more effectively.

Key Results

  • Subjective Feedback: Users reported significantly higher rapport in the mutual gaze condition (M=4.0) compared to the default condition (M=3.5). They also felt the robot understood them better in the mutual gaze condition (M=4.0 vs. M=2.0).
  • Social Perception: The robot was perceived as significantly more "social" in the mutual gaze condition (M=4.5 vs. M=3.5).
  • Task Performance: In the quiz task, users focused more attention on the robot in the mutual gaze condition (60%) compared to the default (39%), and this difference was statistically significant (p=0.020). Interaction time for successfully completing the quiz was significantly shorter in the mutual gaze condition (120.5s) than the default (202.7s).
  • Emotional Response: Users felt significantly less nervous talking to the robot in the mutual gaze condition (M=2) compared to the default (M=4).

Limitations and Future Work

  • Limited Semantic Integration: The current gaze model does not incorporate semantic information from the dialogue, meaning gaze aversion directions are assigned randomly. Future work could explore integrating gaze with speech content for more contextually appropriate behavior.
  • Focus on Gaze Alone: This study primarily focused on the effect of gaze in isolation. Future research should incorporate other modalities, such as speech and gestures, to create more holistic and natural HRI.
  • Understanding Arousal: While the paper speculates that prolonged gaze in the default condition might increase arousal and cognitive load, further studies using biosignals are needed to verify this.
  • Data-Driven Approaches: The authors suggest investigating data-driven approaches to learn gaze models from human interaction data, which could lead to even more sophisticated and adaptive gaze behaviors.

Summary updated

BibTeX
@inproceedings{zhang2017look,
  author = {Yanxia Zhang and Jonas Beskow and Hedvig Kjellström},
  title = {{Look but Don't Stare: Mutual Gaze Interaction in Social Robots}},
  booktitle = {ICSR},
  year = {2017},
  doi = {10.1007/978-3-319-70022-9\_55},
}
Figure