Abstract
Deep learning (DL) models are traditionally deployed on high-performance cloud servers. However, the rise of real-time applications in computer vision and medical diagnostics necessitates a shift toward edge computing. This survey explores the methodologies for deploying DL on resource-constrained devices, covering hardware categorization, model compression, and lightweight architecture design. By processing data locally, Edge DL addresses latency, bandwidth, and privacy concerns, particularly in critical healthcare scenarios.
Introduction
The proliferation of Convolutional Neural Networks (CNNs) has revolutionized computer vision, yet their deployment remains challenging due to high computational demands. Sending high-resolution data, such as medical scans or live video feeds, to the cloud introduces significant latency and privacy risks. Edge computing mitigates these issues by processing data at the source.
This survey provides a structured overview of the Edge DL landscape, as illustrated in the figure above, ranging from fundamental hardware paradigms to specialized applications in medical imaging and autonomous systems.
Hardware Paradigms and Categorization
A key contribution of this work is the classification of edge devices into three tiers based on their computational capabilities:
- Low-end: Microcontrollers like ARM Cortex M or SparkFun boards.
- Medium-range: Single-board computers and mobile devices, such as Raspberry Pi, NVIDIA Jetson Nano, and modern smartphones.
- High-performance: Advanced edge modules like the NVIDIA Jetson AGX Orin.
As shown in the hierarchy, the "Edge" layer encompasses a diverse range of devices—from IoT sensors and cameras to autonomous vehicles—each presenting unique constraints for AI deployment.
Enabling Efficient Edge Deep Learning
To bridge the gap between massive DL models and tiny devices, several optimization strategies are employed:
Model Compression
Techniques like Network Pruning remove redundant weights or neurons, while Quantization reduces weight precision (e.g., from 32-bit floats to 8-bit integers). These methods significantly reduce memory footprints and latency without substantial accuracy loss.
Lightweight Architectures
Instead of shrinking large models, researchers are designing inherently efficient networks. Examples include MobileNet (utilizing depthwise separable convolutions), EfficientNet (using compound scaling), and mobile-optimized Vision Transformers like MobileViT.
Neural Architecture Search (NAS)
NAS automates the design of hardware-aware networks. Frameworks like ProxylessNAS factor in specific device constraints (latency, memory) during the search phase to find the optimal architecture for a given target platform.
Applications in Medical Diagnostics
Edge DL is particularly transformative in healthcare. Traditional Computer-Aided Diagnosis (CAD) systems are often bulky; Edge DL enables portable, real-time diagnostics:
- Smart Endoscopy: Real-time detection of gastrointestinal polyps during procedures.
- Portable Ultrasound: Devices like the Butterfly iQ use on-device AI to guide non-experts in the field.
- Privacy Preservation: By processing Patient Health Information (PHI) locally, Edge DL simplifies compliance with regulations like HIPAA and GDPR.
Key Results and Performance
The survey highlights that Edge DL models can achieve clinical-grade performance. For instance, a lightweight CNN for mobile devices reached 97.9% accuracy in detecting pulmonary nodules from CT scans. Furthermore, model compression techniques like channel pruning can reduce FLOPs by 2x to 3x while maintaining over 95% of the original model's accuracy.
Future Directions
Despite its promise, the field faces challenges in hardware heterogeneity and the memory-intensive nature of on-device training. Future research is directed toward:
- Explainable AI (XAI): Developing lightweight frameworks to provide interpretable results for clinicians.
- Adaptive Architectures: Networks that adjust complexity based on real-time battery or thermal constraints.
- Federated Learning: Collaborative training across devices without sharing raw sensitive data.
Conclusion
Edge DL represents a paradigm shift in AI deployment, moving intelligence closer to the data source. By combining hardware-aware design with advanced compression techniques, Edge DL enables a new generation of responsive, private, and efficient applications in computer vision and medical diagnostics.
Summary updated