Stereo Camera Based Virtual Cane System with Identifiable Distance Tactile Feedback for the Blind

Donghun Kim, Kwangtaek Kim, Sangyoun Lee2014 · DOI

TL;DR

This paper introduces a novel virtual cane system for the visually impaired that uses a stereo camera to track a user's finger pointing gesture and provide real-time obstacle detection and distance estimation. The system enhances usability by integrating a unique tactile feedback mechanism that uses identifiable vibration frequencies to convey distance information, offering a more intuitive and robust navigation aid.

What Problem Does This Paper Address?

Traditional white canes, while essential, have limitations such as a limited sensing range, discomfort, and the need for extensive training. Existing virtual cane systems often rely on complex sensors or provide less intuitive feedback. This paper aims to bridge this gap by developing a non-intrusive, user-friendly virtual cane system that leverages advanced computer vision for precise finger tracking and obstacle detection, combined with a novel, perceptually identifiable tactile feedback system to convey crucial environmental information to the blind user. The goal is to provide a more natural and effective way for visually impaired individuals to navigate their surroundings.

Methods

The system, named Visual Information Delivery Assistant (VIDA), operates in two main stages:

  • Visual Information Delivery Assistant (VIDA):
    • Hand and Finger Detection: Employs a fusion of static and dynamic background subtraction techniques, combined with skin-color segmentation in the YUV and YIQ color spaces, to robustly detect the user's pointing finger even in dynamic and cluttered environments.
  • 3D Pointing Direction Estimation: Utilizes stereo vision principles and hand shape analysis (convexity and geometric features) to estimate the 2D pointing direction, which is then extended to 3D using camera calibration parameters.
  • Object Detection and Distance Calculation: Identifies objects within the estimated pointing trajectory by classifying 3D points and extracting Regions of Interest (ROIs). The distance to detected objects is then computed using stereo camera geometry.
  • Tactile Feedback Interface:
    • Identifiable Frequency Selection: Conducted identification experiments with human participants to determine distinct vibration frequencies that can be reliably perceived and distinguished on the index finger. This led to the selection of 10 Hz, 100 Hz, and 600 Hz.
  • Dual Actuator Design: Integrates two actuators: one on the index finger for distance feedback (mapping different distances to the identified frequencies) and another on the wrist for guidance feedback (alerting the user when their hand is out of the camera's view).

Datasets

Not explicitly discussed in the paper. The system is evaluated using real-world scenarios and experiments with human participants, with ground truth data generated using a laser pointer.

Key Insights and Ideas

  • Finger Pointing as an Intuitive Interface: Using a simple finger pointing gesture as the primary interaction method for navigation is a natural and less intrusive approach for visually impaired users.
  • Robust Hand and Finger Detection: Combining background subtraction with color segmentation in YUV/YIQ spaces provides a robust solution for detecting pointing gestures even in challenging dynamic environments.
  • Perceptually Identifiable Tactile Feedback: The core innovation lies in designing tactile signals based on human perception rather than arbitrary choices. This ensures that the feedback is not just present but truly informative and distinguishable, crucial for safety and navigation.
  • Dual-Feedback System: The integration of both distance feedback (index finger) and guidance feedback (wrist) provides a comprehensive and layered approach to informing the user about their environment.
  • Frequency as a Key Parameter for Tactile Feedback: Focusing on frequency identification for tactile signals proved more effective than amplitude due to hardware limitations and user perception, leading to a more reliable system.

Key Results

  • The VIDA system achieves accurate finger tracking and 3D pointing estimation in real-time, running at 6 frames per second.
  • Object detection was demonstrated to be accurate, with reported distance errors of around 6-8 pixels for various object sizes.
  • The pointing algorithm showed good accuracy with a laser pointer as ground truth, with average errors ranging from 6.98 cm at 1m to 25.58 cm at 3m.
  • The system demonstrated robustness against varying illumination conditions and occlusions (e.g., extra hands, faces).
  • The identification experiments successfully identified three distinct tactile frequencies (10 Hz, 100 Hz, 600 Hz) that participants could reliably distinguish.

Limitations and Future Work

  • Frame Rate: The current frame update rate of 6 Hz might be a limitation for faster walkers. Future work aims to improve this.
  • User Studies: While experiments were conducted, the paper suggests conducting user studies with visually impaired people to further evaluate the system's effectiveness and usability in real-world scenarios.
  • Tactile Signal Design: The authors are interested in exploring other parameters for tactile signals, such as amplitudes and complex wave forms, through further identification and psychophysical experiments.
  • Completeness of Navigation: While the system provides obstacle detection and guidance, a complete navigation solution for the blind would likely involve integration with other assistive technologies or more sophisticated path planning.

Summary updated

BibTeX
@misc{kim2014stereo,
  author = {Donghun Kim and Kwangtaek Kim and Sangyoun Lee},
  title = {{Stereo Camera Based Virtual Cane System with Identifiable Distance Tactile Feedback for the Blind}},
  year = {2014},
  doi = {10.3390/s140610412},
}

Loading related content...

Figure