Enabling Computer Vision Driven Assistive Devices for the Visually Impaired via Micro-architecture Design Exploration

Linda Wang, Alexander WongarXiv 2019 · arXiv

TL;DR

This paper introduces a method to optimize object detection models for edge devices, specifically targeting an AI assistant for the visually impaired called OLIV. By treating neural network micro-architecture design as a numerical optimization problem, the authors find an optimal balance between accuracy, speed, and model size. The resulting MobileNetV2-SSD model runs locally on-device, preserving user privacy and reducing costs without sacrificing critical performance.

The Challenge: Edge-Based Assistive Vision

Visually impaired individuals can greatly benefit from computer vision tools that help them locate displaced objects in their environment, such as finding a misplaced mouse on a desk. However, state-of-the-art object detection models are typically massive and computationally hungry, requiring powerful GPUs.

System architecture of the OLIV assistive device, illustrating the interaction between the user, speech module, and on-device object detection.

This creates a dilemma for assistive devices: running these models in the cloud raises significant privacy concerns (streaming video of a user's home) and ongoing server costs, while running them locally on affordable edge devices is difficult due to hardware constraints. The authors tackle the challenge of designing a compact, fast, and accurate object detection network that can run entirely on-device for the OLIV system (an AI-powered assistant for object localization).

Methods: Micro-architecture Design Exploration

The authors employ a "micro-architecture design exploration" strategy to optimize a MobileNetV2-SSD object detection network. Instead of manually guessing the best network size, they formulate the architecture search as a numerical optimization problem. They specifically tune two key hyperparameters:

  • Width multiplier ($\alpha$): Scales the width of the network uniformly by multiplying the number of input and output channels at each layer.
  • Resolution multiplier ($\rho$): Scales the input image resolution.

To evaluate the trade-offs of different $\alpha$ and $\rho$ combinations, they use a modified NetScore objective function. NetScore quantitatively balances a network's accuracy, architectural complexity (number of parameters), and computational complexity (CPU running time). Because accuracy is paramount for a reliable assistive device, the authors heavily weight the accuracy metric compared to model size and speed in their objective function.

Datasets

The research utilizes MSCOCO-OLIV, a custom 21-class subset of the standard COCO2017 dataset. This subset specifically isolates indoor objects relevant to the daily lives of visually impaired users, rather than the broader, general-purpose categories found in the full COCO dataset.

Key Insights and Results

The optimization process identified the absolute optimal MobileNetV2-SSD configuration for the OLIV system: a width multiplier of 1.3 and an input resolution of 224.

NetScore surface plot showing the optimization landscape and the identified peak performance point at (1.3, 224).

Key findings from the exploration include:

  • Diminishing Returns in Scaling: Increasing the width multiplier and input resolution generally improves Mean Average Precision (mAP) and increases CPU run-time. However, the authors found a clear inflection point: after a width multiplier of 1.15 and a resolution of 220, accuracy gains severely plateau, while computational costs continue to rise linearly.
  • Objective-Driven Search: By mathematically defining the ideal trade-off via the modified NetScore, the researchers could systematically pinpoint the exact hyperparameter pairing that maximizes utility for edge-based assistive tech. This configuration achieved the highest modified NetScore of 68.7.
  • Privacy through Efficiency: By successfully shrinking the model to run on a local CPU without losing critical accuracy, the system inherently protects user privacy—a major hurdle for the adoption of always-on cameras in domestic environments.

Limitations and Future Work

The paper is a brief exploration and does not include a dedicated limitations section. For instance, it does not detail the specific hardware used for the CPU benchmarking or how the system performs in real-world, poorly lit environments. However, the authors note that this micro-architecture exploration approach has the potential to be applied to other assistive devices beyond OLIV, suggesting a future direction of generalizing this optimization pipeline to different hardware constraints and novel use cases.

Summary updated

Abstract

Recent improvements in object detection have shown potential to aid in tasks where previous solutions were not able to achieve. A particular area is assistive devices for individuals with visual impairment. While state-of-the-art deep neural networks have been shown to achieve superior object detection performance, their high computational and memory requirements make them cost prohibitive for on-device operation. Alternatively, cloud-based operation leads to privacy concerns, both not attractive to potential users. To address these challenges, this study investigates creating an efficient object detection network specifically for OLIV, an AI-powered assistant for object localization for the visually impaired, via micro-architecture design exploration. In particular, we formulate the problem of finding an optimal network micro-architecture as an numerical optimization problem, where we find the set of hyperparameters controlling the MobileNetV2-SSD network micro-architecture that maximizes a modified NetScore objective function for the MSCOCO-OLIV dataset of indoor objects. Experimental results show that such a micro-architecture design exploration strategy leads to a compact deep neural network with a balanced trade-off between accuracy, size, and speed, making it well-suited for enabling on-device computer vision driven assistive devices for the visually impaired.
BibTeX
@misc{wang2019enabling,
  author = {Linda Wang and Alexander Wong},
  title = {{Enabling Computer Vision Driven Assistive Devices for the Visually Impaired via Micro-architecture Design Exploration}},
  year = {2019},
  eprint = {1905.07836},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1905.07836},
}
Figure