A low-cost tele-presence wheelchair system

Jiajun Shen, Bin Xu, Mingtao Pei, Yunde Jia

TL;DR

This paper introduces a low-cost tele-presence wheelchair that allows remote caregivers to "push" a patient’s wheelchair through a touch-screen interface as if they were physically there. By combining a standard electric wheelchair with a custom sensor module and semi-autonomous navigation, the system provides both mobility for the user and social connection for the remote operator.

What Problem Does This Paper Address?

Elderly and disabled individuals often face two simultaneous challenges: limited physical mobility and social isolation (geriatric depression). While "Intelligent Wheelchairs" (IWs) exist to help with mobility, they are often prohibitively expensive, require heavy training, and don't address the need for social presence. Conversely, "Tele-presence Robots" allow remote social interaction but don't typically assist the person on the other end with their own mobility.

The authors aim to bridge this gap by creating a system that is:

  1. Low-cost: Using an add-on module for commercial electric wheelchairs.
  2. Intuitive: Reducing the "cognitive load" of remote operation.
  3. Collaborative: Using semi-autonomous navigation to help remote operators navigate tricky spaces like doorways without taking full control away from the human.

Methods

The system consists of three main components: a commercial electric wheelchair, a tele-presence interaction module, and a Touchable Live Video Image based User Interface (TIUI).

  • The Imaging System: To give the remote operator a clear view, the authors mounted two cameras: a Forward-Facing (FF) camera for social interaction and a Down-Facing (DF) camera with a wide-angle lens for navigation. They used SURF (Speeded Up Robust Features) to stitch these into a single "Forward-Down-Facing" (FDF) live stream, preventing the confusion of switching between multiple windows.
  • The TIUI Interface: Instead of a traditional joystick, the remote operator uses a tablet. They can "push" the wheelchair by touching the lower part of the video (the ground) or interact with objects (like doors or elevators) by touching the upper part of the video.
  • Semi-Autonomous Navigation: Navigating a remote robot through a narrow door is notoriously difficult due to lag and perspective issues. The system uses edge and corner detection to identify doorways and generates safe trajectories using Bézier Curves. This is a "human-in-the-loop" approach: the operator indicates the intent to move, and the system smooths out the path to avoid hitting the frame.

Datasets

The authors did not use standard public datasets. Instead, they evaluated the system using:

  • User Experience Study: 20 volunteers (ages 18-25) tested the system in a simulated living environment containing sofas, tables, and chairs.
  • Navigation Benchmark: A custom set of ten different doors and four specific doorway configurations (ranging from a single door to complex scenes with more than three doors) to test the robustness of the computer vision algorithms.

Key Insights and Ideas

  • The "Virtual Track-Point": By placing a red circle on the lower part of the video stream, the authors created a virtual joystick that feels like "pushing" the chair from behind. This mapping of touch-to-motion is more natural for novice users than abstract controls.
  • Stitching for Context: The paper highlights that having two separate video feeds is distracting. By stitching the FF and DF views, the operator maintains "situational awareness"—they can see the person they are talking to while simultaneously seeing the floor they are navigating.
  • Assistance over Autonomy: Rather than trying to build a fully autonomous robot (which is expensive and prone to failure in edge cases), the authors focused on "collaborative control." The robot only assists during high-difficulty maneuvers, like centering itself in a doorway.

Key Results

  • Efficiency: Remote operators were nearly as fast as people pushing the wheelchair in person. The average time to complete a course was 34.8 seconds via tele-presence versus 33.1 seconds in person.
  • Usability: 90% of participants found the TIUI interface user-friendly and easy to learn.
  • Navigation Success: The semi-autonomous system performed well in simple scenes (86.6% "perfect" navigation for single doors). However, performance dropped as the number of doors in the view increased, showing that visual clutter remains a challenge for the detection algorithms.
  • Time Savings: The navigation assistance was most valuable at "high offset angles" (when the wheelchair isn't lined up with the door). In these cases, the semi-autonomous mode was significantly faster than pure manual remote control.

Limitations and Future Work

The authors acknowledge several areas for improvement:

  • Complexity Scaling: The vision system struggles when the environment becomes too crowded with multiple doors or complex layouts.
  • Real-World Testing: The current study used healthy young volunteers in a controlled lab; future work needs to involve the actual target demographic (elderly and disabled users) in unpredictable home environments.
  • Bandwidth/Latency: While not deeply explored, the reliance on live video stitching suggests that high-speed internet is a prerequisite, which may not be available in all care settings.
  • Algorithm Refinement: They plan to improve the semi-navigation algorithms to further reduce the "operator overload" during long-distance travel.

Summary updated

Abstract

This paper presents the architecture and implementation of a tele-presence wheelchair system based on tele-presence robot, intelligent wheelchair, and touch screen technologies. The tele-presence wheelchair system consists of a commercial electric wheelchair, an add-on tele-presence interaction module, and a touchable live video image based user interface (called TIUI). The tele-presence interaction module is used to provide video-chatting for an elderly or disabled person with the family members or caregivers, and also captures the live video of an environment for tele-operation and semi-autonomous navigation. The user interface developed in our lab allows an operator to access the system anywhere and directly touch the live video image of the wheelchair to push it as if he/she did it in the presence. This paper also discusses the evaluation of the user experience.
BibTeX
@inproceedings{shen2016low,
  author = {Jiajun Shen and Bin Xu and Mingtao Pei and Yunde Jia},
  title = {{A low-cost tele-presence wheelchair system}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2016},
  url = {https://openreview.net/forum?id=P0qASvQKWV},
}

Loading related content...

Figure