For automated vehicles (AVs) to truly deliver on the promise of mobility independence, they must be accessible by design, not as an afterthought. A critical part of this is the vehicle’s ability to "see" and understand the specific needs of a passenger before they even reach the door.
A new paper from researchers at the University of California, Merced, addresses a vital gap in this pipeline: distinguishing between different wheelchair configurations to help AVs anticipate boarding and securement requirements.
TL;DR
The authors introduce the Wheelchair Configuration Image Dataset (WCID), a new benchmark for classifying manual versus powered wheelchairs. By evaluating several deep learning architectures, they demonstrate that while modern models can achieve high accuracy, they remain sensitive to image blur and benefit significantly from "selective deferral"—allowing the system to abstain from a prediction when confidence is low.
What Problem Does This Paper Address?
Current computer vision datasets for mobility aids are often limited to simple detection—identifying that a wheelchair is present. However, for an AV, knowing what kind of wheelchair is approaching is essential.
Powered wheelchairs are generally heavier and have larger footprints than manual ones, requiring different floor space (up to 870 mm by 1508 mm) and securement protocols. Furthermore, wheelchairs are highly individualized; variations in frame geometry, seating systems, and accessories, combined with the presence of occupants or bags, make them a "visually non-uniform" object class. This paper seeks to determine how accurately visual classifiers can recover these configurations from photographs to initialize boarding checklists or prompt human assistance.
Methods
The researchers benchmarked four established model families:
- ResNet-50: A standard residual baseline for transfer learning.
- EfficientNet-B0: A model optimized for edge deployment (relevant for vehicle-side compute).
- ConvNeXt-Tiny: A modern "pure" convolutional network designed to compete with Vision Transformers.
- SigLIP 2: A frozen vision-language encoder evaluated with a linear probe to see how well general-purpose pre-trained features perform on this niche task.
Beyond standard classification, the team employed selective classification. Instead of forcing a prediction on every frame, the model uses its maximum softmax probability as a confidence measure. If the confidence is too low, the system "defers" the decision, potentially waiting for a better camera angle or human intervention. They also conducted a robustness analysis using Gaussian blur to simulate real-world conditions like rain, fog, or motion blur.
Datasets
The core contribution is the Wheelchair Configuration Image Dataset (WCID).
- Composition: 467 face-blurred images across 192 "capture groups" (images of the same chair/user from different angles).
- Annotations: Each image is labeled for manual vs. powered status, subtype (e.g., tilt-recline, bariatric, negative camber), occupancy, viewpoint, lighting, and occlusion.
- Training: Models were trained on a separate set of 4,362 external images from OpenImages and OpenVerse.
- Partitioning: The authors used "group-exclusive partitioning," ensuring that images of the same physical wheelchair do not appear in both the training and evaluation sets, preventing the models from simply memorizing specific devices.
Key Insights and Ideas
- Configuration Matters for Safety: Distinguishing powered chairs (via battery housings or joysticks) from manual chairs provides the necessary context to initialize vehicle boarding and securement checklists before the passenger arrives.
- The Power of Deferral: In safety-critical AV applications, a wrong guess is worse than no guess. The study shows that by deferring the 30% of images where the model is least confident, accuracy jumps significantly.
- High-Frequency Cues are Fragile: Models rely heavily on fine details like wheel spokes, hand rims, and joystick boxes. When these are obscured by blur, performance tanks, especially for convolutional models.
- Manual vs. Powered Bias: Every model tested achieved lower recall for powered wheelchairs than for manual ones, suggesting that powered chairs may have more visual overlap with other objects or more varied configurations that are harder to learn.
Key Results
- Top Performer: ConvNeXt-Tiny achieved the highest mean accuracy at 92.1% and a Macro-F1 score of 0.892.
- Selective Deferral Gains: By retaining only the most confident 70% of predictions, ConvNeXt-Tiny’s accuracy increased from 92.1% to 98.0%.
- Blur Sensitivity: Performance dropped sharply under Gaussian blur. For example, at a blur severity of $\sigma=4$, ConvNeXt-Tiny’s Macro-F1 dropped to 0.756 (when blurred before resizing) and as low as 0.598 (when blurred at model-input resolution).
Limitations and Future Work
The authors acknowledge that the benchmark is limited by its scale (467 images) and a potential bias toward web-sourced imagery. Currently, the models are "closed-set," meaning they might assign high confidence to non-wheelchair mobility aids they haven't seen before.
Future work aims to expand the dataset with multi-site physical device captures, co-design taxonomies directly with wheelchair users, and integrate "open-set" detection to better handle unfamiliar mobility devices in the wild.
Summary updated