E-Gaze Glasses: Simulating Natural Gazes for Blind People

Shi Qiu, Siti Aisyah Anas, Hirotaka Osawa, Matthias Rauterberg, Jun HuTEI 2016 · DOI

TL;DR

This paper introduces E-Gaze glasses, a wearable assistive device that simulates natural eye contact for blind individuals in face-to-face conversations. By using an eye-tracking system and a turn-taking model, E-Gaze aims to enhance social interaction and engagement for the visually impaired.

What Problem Does This Paper Address?

In social interactions, gaze and eye contact play a crucial role in establishing trust, signaling attention, and facilitating engagement between people. Sighted individuals unconsciously use these visual cues to gauge interest, understand emotional states, and manage the flow of conversation. However, for blind individuals, these vital nonverbal signals are inaccessible, making face-to-face communication challenging and potentially hindering their ability to connect with sighted conversation partners. This paper addresses the gap in assistive technology by proposing a system that bridges this visual communication divide.

Methods

The core of the E-Gaze system is a wearable device, designed as glasses, that incorporates an eye-tracking system and an OLED display to simulate eye gestures. The system operates based on two primary modes: "while speaking" and "while listening."

  • Eye Tracking: An external Eye Tribe Tracker is used to monitor the gaze of the sighted conversation partner. This tracker captures eye gaze coordinates with an average efficiency of 0.5 to 1 degree of visual angle.
  • Speech Detection: A speech detection module (currently under development) is intended to identify when the blind user is speaking or listening.
  • "While Speaking" Mode: When the blind user is speaking, the system aims to simulate a "look away" gaze, allowing the user to concentrate on their speech.
  • "While Listening" Mode: When the blind user is listening, the system aims to simulate "eye contact" by having the E-Gaze glasses look towards the sighted person. This is triggered when the sighted person's gaze is detected within a specific area.
  • Turn-Taking Model: The gaze behavior is designed to align with a turn-taking model, where gaze shifts signal the end of a speaker's turn and invite the other person to speak.
  • Eye Contact Simulation: The system simulates eye contact by having the E-Gaze glasses look back at the sighted person when the sighted person looks at the E-Gaze.
  • Avoiding Long Gaze: A feature is included to simulate looking away if the sighted person maintains a prolonged gaze, aiming to create a more comfortable interaction.
  • Attention State: In a conceptual design, the simulated eyes could widen when the blind person's heart rate increases, indicating an "attention state."

The system is built using an Arduino microcontroller and Bluetooth for wireless communication, driving two uOLED displays. The programming is done in Java using an Eclipse IDE.

Datasets

Not discussed in the paper. The paper focuses on the conceptual design and prototype implementation rather than large-scale data collection or training.

Key Insights and Ideas

  • Bridging the Visual Communication Gap: The fundamental idea is that simulating natural gaze can significantly improve social interaction for blind individuals, making them feel more connected and engaged in conversations.
  • Turn-Taking as a Foundation for Gaze: Linking gaze behavior directly to conversational turn-taking provides a structured and intuitive way to simulate natural interaction, rather than relying on random or static gaze patterns.
  • The Importance of "Eye Contact" for the Blind: The research highlights that even simulated "eye contact" can have a profound impact on the perceived engagement and comfort of both blind and sighted individuals in a conversation.
  • Wearable Technology for Social Inclusion: The development of a glasses-based wearable device makes the technology discreet and integrated into a common personal item, increasing its potential for practical use.

Key Results

The paper presents a functional prototype and a small test that verified the system's ability to detect human gaze and produce intended animation outputs. A prior user study with 20 participants evaluated conceptual features based on usefulness, efficiency, and interest, with more detailed results reported elsewhere. The current working-progress paper refines two design features (eye contact simulation and avoiding state) based on the turn-taking gaze model.

Limitations and Future Work

  • Speech Detection Module: The speech detection module for identifying speaking and listening states is currently unfinished. This is a critical component for the full implementation of the turn-taking model.
  • Full Functionality: The current implementation focuses on following the sighted person's gaze. The full turn-taking gaze behavior is still under development.
  • User Study: While a prior conceptual study was conducted, the paper describes a working-progress prototype, implying that further user studies with the refined prototype are needed to validate its effectiveness in real-world scenarios.
  • Future Work: The authors explicitly state that in their future work, they will develop the speech detection module for the E-Gaze system based on the turn-taking model. They also aim to develop a fully functional prototype.

Summary updated

BibTeX
@inproceedings{qiu2016e,
  author = {Shi Qiu and Siti Aisyah Anas and Hirotaka Osawa and Matthias Rauterberg and Jun Hu},
  title = {{E-Gaze Glasses: Simulating Natural Gazes for Blind People}},
  booktitle = {TEI},
  year = {2016},
  doi = {10.1145/2839462.2856518},
}
Figure