Abstract
We develop an edge-assisted object recognition system with the aim of studying the system-level trade-offs between end-to-end latency and object recognition accuracy. We focus on developing techniques that optimize the transmission delay of the system and demonstrate the effect of image encoding rate and neural network size on these two performance metrics. We explore optimal trade-offs between these metrics by measuring the performance of our real time object recognition application. Our measurements reveal hitherto unknown parameter effects and sharp trade-offs, hence paving the road for optimizing this key service. Finally, we formulate two optimization problems using our measurement-based models and following a Pareto analysis we find that careful tuning of the system operation yields at least 33% better performance for real time conditions, over the standard transmission method.
Introduction
Mobile devices are often too underpowered to run state-of-the-art deep learning models locally with low latency. The standard solution is "edge offloading": sending images to a nearby server for processing. However, this introduces a multi-dimensional trade-off between image quality, model complexity, and network overhead. Standard protocols like TCP and mobile power-saving features often add significant, overlooked delays that hinder real-time performance. This paper quantifies these effects and provides a framework for tuning system parameters to find the "Pareto-optimal" configuration for real-time object recognition.
System Architecture and Methodology
The researchers built a complete experimental testbed consisting of various Android handsets (Pixel 2, Galaxy S8, Huawei P10 Lite) and an edge server equipped with an NVIDIA RTX 2080Ti GPU. The system follows a pipeline of image capture, encoding, wireless transmission, server-side decoding, and neural network inference.
Key technical interventions included:
- Network Optimization: Replacing standard TCP with "UDP packet bursting" to avoid slow-start congestion control and acknowledgement delays.
- NIC Power Management: Identifying and mitigating the latency added by the handset's Network Interface Controller (NIC) sleep modes.
- System Modeling: Creating quadratic mathematical models for four delay components: encoding ($T_{enc}$), transmission ($T_{tx}$), decoding ($T_{dec}$), and deep learning inference ($T_{dl}$).
Latency Bottlenecks and Optimization
A major finding of the study is the impact of the Wi-Fi NIC power-save mode, which the authors describe as a "silent killer" of latency. The handset's NIC frequently enters a sleep mode to save battery, adding approximately 5ms of "wake-up" latency to every transmission. By using background traffic to keep the NIC awake, the team reduced transmission delay by up to 60%.
Furthermore, the study found that for edge-assisted vision, the reliability of TCP comes at too high a cost. Sending images as back-to-back UDP bursts was 3x faster than TCP because it avoided contention caused by return ACKs and the overhead of standard congestion control mechanisms.
Key Results
- Transmission Speedup: By combining UDP bursting and disabling NIC power-save, transmission time was reduced by 65% (from ~15-20ms down to ~2-6ms).
- Precision and Frame Rate Gains: Under real-time constraints (30 fps), the optimized system provided 33% better precision than a "vanilla" setup. In scenarios where lower accuracy is acceptable, the system achieved up to a 93% higher frame rate.
- Real-time Feasibility: The system successfully maintained satisfactory precision (0.5 AP) even at 40 fps, proving that edge-assisted AR and video analytics are viable with careful tuning.
Conclusion
This measurement-driven analysis demonstrates that significant performance gains in edge-assisted vision are possible through low-level network and power management optimizations. While the study was conducted in a controlled high-performance Wi-Fi environment, it paves the way for future work exploring performance in congested networks and the specific battery impacts of keeping mobile radios in an active state.
Summary updated