AVD: A Multi-Sensor Teleoperated Wheelchair Platform with Safety-Aware Control

Namai Chandra, Madhur Thareja, Boby GeorgearXiv 2026 · arXiv

TL;DR

AVD (Autonomous Vehicle Drive) is a low-cost, safety-first teleoperation architecture for electric wheelchairs that prioritizes dependability over full autonomy. By using a dual-controller "heartbeat" system and a strict control hierarchy, it ensures the vehicle always fails into a safe state, even during sensor dropouts or network lag.

What Problem Does This Paper Address?

For many individuals with severe motor or visual impairments, the standard proportional joystick—the "gold standard" of powered wheelchairs for decades—is an insurmountable barrier. While academic research has flirted with full autonomy (SLAM, path planning, etc.), these systems are often too expensive, complex, and fragile for real-world deployment.

The authors identify a critical "missing middle": a system that allows a remote human (a caregiver or specialist) to navigate the chair while ensuring the on-board hardware remains a "dependable" guardian. The core challenge isn't just making the chair move; it’s defining exactly what the chair should do when a sensor fails, the internet drops, or a vibration shakes a wire loose.

Methods

The researchers built AVD using a modular, off-the-shelf hardware stack (totaling roughly $335) and a novel safety-oriented design framework.

The Dual-Domain Architecture

The system splits computation into two distinct "fault domains":

  • Domain A (Primary Controller): A Raspberry Pi 5 running a general-purpose OS. It handles the "heavy lifting"—streaming four camera feeds, processing 360° LiDAR data, and managing the web-based mission control dashboard. Because it runs a complex OS, it is considered "non-deterministic" (prone to hangs or crashes).
  • Domain B (Safety Controller): An ESP32 microcontroller running a bare-metal loop. It has no network stack and no filesystem. Its sole job is to drive the motors and enforce safety logic.

The Heartbeat and Hierarchy

The two domains are linked by a heartbeat-driven watchdog. The Primary Controller must send a "heartbeat" every 100ms. If the Safety Controller misses five beats (500ms), it automatically shifts the chair to neutral. If the silence continues for 1500ms, it engages the brakes.

Control authority follows a strict, static hierarchy:

  1. Manual E-Stop: Physical override.
  2. On-board Sensing: "Stop-only" authority. Sensors can halt the chair if an obstacle is detected within 50cm, but they can never initiate movement.
  3. Occupant Joystick: Local manual override always pre-empts remote commands.
  4. Teleoperator Commands: The lowest priority, honored only if the link is "fresh."

Datasets

Rather than using standard machine learning datasets, the authors developed an experimental characterization methodology to stress-test their hardware:

  • LiDAR Degradation Campaign: A 2,025-scan study evaluating performance under water mist, airborne dust, and direct aperture occlusion (simulating a dirty lens).
  • Attitude Characterization: Measuring IMU reliability across static inclines up to 47°.
  • Latency Benchmarking: Measuring the "real-world" acquisition time for I2C, USB, and CSI sensor interfaces.

Key Insights and Ideas

  • "Stop-Only" Authority: This is a brilliant simplification. By stripping the sensing layer of the ability to drive the chair, the authors eliminate a massive class of "ghost-in-the-machine" errors where a sensor glitch might cause a chair to lunge forward.
  • Complementary Failure Modes: During testing, the team logged a real-world failure where high vibrations caused an I2C dropout on the IMU. Crucially, the LiDAR and cameras remained functional. This highlights why "late fusion" or independent sensor channels are safer than tightly coupled systems where one failure might blind the entire stack.
  • The Latency Argument: The paper quantifies why we shouldn't trust cameras for automatic braking. Camera latency was found to be 27x more jittery (higher standard deviation) than LiDAR or IMU data, making the latter the only logical choice for "heartbeat" safety decisions.

Key Results

  • LiDAR Robustness: Even in the worst-case scenario (aperture occlusion), the LiDAR retained 76.6% of its clear-baseline yield. Most importantly, no scan in the 2,025-scan campaign lost the forward sector entirely.
  • System Latency: End-to-end command latency (from remote dashboard to motor actuation) was observed between 200ms and 350ms over a standard 2.4 GHz link—well within the usable range for low-speed (<2 km/h) operation.
  • Cost Efficiency: The entire instrumented package costs ~$335, proving that dependable assistive tech doesn't require "self-driving car" budgets.

Limitations and Future Work

The authors are refreshingly honest about the prototype's current boundaries:

  • The "Real Hardware" Gap: The system was tested on a hobby-grade chassis with an H-bridge. While the interface is designed to mimic a commercial joystick, it hasn't been bench-tested on a proprietary commercial wheelchair power module yet.
  • Planar Blindness: The 2D LiDAR only sees a single horizontal slice. It is "structurally blind" to drop-offs (like stairs) or overhead obstacles (like a table edge).
  • Human Factors: No formal trials with individuals with disabilities have been conducted yet.
  • Next Steps: The team plans to move to "kilometre-scale" field deployments and perform formal fault-injection testing to validate the safety-controller policies.

Summary updated

Abstract

Electric wheelchairs predominantly rely on manual joystick interfaces, which present critical barriers to independent mobility for individuals with severe motor and visual impairments. This paper presents AVD (Autonomous Vehicle Drive), a dependable teleoperation architecture for assistive mobility that unifies perception, communication, embedded control, and remote supervision in a single low-cost platform built entirely from commercially available hardware. The central engineering contribution is a safety-oriented design framework: control authority follows a static hierarchy, a manual emergency stop and the occupant's joystick pre-empt remote commands, and the on-board sensing layer has stop-only authority. The compute design partitions primary and safety functions and specifies a heartbeat-driven safe-state transition for the fault cases considered. The sensing stack this framework depends on is characterized against measured data: per-channel acquisition latency, a 2,025-scan LiDAR degradation campaign spanning five target configurations and a control under water mist, airborne dust, and direct aperture occlusion, and attitude measurements across static inclines to 47 degrees. The lowest pooled environmental-condition mean retained 76.6% of the clear-baseline forward-sector yield, and no scan lost the sector completely. A logged vibration-induced inertial dropout occurred while LiDAR and camera logging continued, illustrating complementary channel behavior. The result is a documented prototype and an explicit validation path for a deployable wheelchair implementation.
BibTeX
@misc{chandra2026avd,
  author = {Namai Chandra and Madhur Thareja and Boby George},
  title = {{AVD: A Multi-Sensor Teleoperated Wheelchair Platform with Safety-Aware Control}},
  year = {2026},
  eprint = {2611.20720},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20720},
}
Figure