Measurement-driven Analysis of an Edge-Assisted Object Recognition System

Apostolos Galanopoulos, Victor Valls, George Iosifidis, Douglas J. Leith

TL;DR

This paper explores the delicate balancing act between end-to-end latency and recognition accuracy in edge-assisted computer vision. By optimizing wireless transmission—specifically by bypassing TCP in favor of UDP bursts and managing mobile NIC power states—the authors achieve a 33% performance boost, enabling high-accuracy object recognition at real-time frame rates (30+ fps).

What Problem Does This Paper Address?

Mobile devices are often too underpowered to run state-of-the-art deep learning models locally with low latency. The standard solution is "edge offloading": sending images to a nearby server for processing. However, this introduces a new set of challenges.

The authors identify a multi-dimensional trade-off:

  1. Image Quality vs. Speed: Higher JPEG compression reduces transmission time but degrades recognition accuracy.
  2. Model Complexity vs. Speed: Larger neural networks (NN) are more accurate but take longer to run on the server.
  3. Network Overhead: Standard protocols like TCP and mobile power-saving features (which put the Wi-Fi chip to sleep) add significant, often overlooked, delays that kill real-time performance.

The paper aims to quantify these effects and provide a framework for tuning system parameters to find the "Pareto-optimal" configuration for any given requirement.

Methods

The researchers built a complete experimental testbed consisting of various Android handsets (Pixel 2, Galaxy S8, Huawei P10 Lite) and an edge server equipped with an NVIDIA RTX 2080Ti GPU.

Key technical interventions included:

  • Network Optimization: They replaced standard TCP with "UDP packet bursting." TCP’s congestion control and acknowledgement (ACK) mechanisms were found to be too slow for small, bursty image transmissions.
  • NIC Power Management: They identified that the handset's Network Interface Controller (NIC) frequently enters a sleep mode to save battery, adding ~5ms of "wake-up" latency to every transmission. They used background traffic to keep the NIC awake, simulating a smarter sleep-mode adaptation.
  • System Modeling: They performed extensive measurements to create quadratic mathematical models for four delay components: encoding ($T_{enc}$), transmission ($T_{tx}$), decoding ($T_{dec}$), and deep learning inference ($T_{dl}$).
  • Pareto Analysis: Using these models, they formulated optimization problems to either maximize precision for a target frame rate or maximize frame rate for a target precision.

Datasets

The study utilized the COCO (Common Objects in Context) dataset, specifically a subset of 5,000 images from the validation set. This dataset is a standard benchmark in computer vision, featuring a wide variety of objects in complex, real-world scenes. Performance was measured using standard Average Precision (AP) and Average Recall (AR) metrics.

Key Insights and Ideas

  • The "Silent Killer" of Latency: The Wi-Fi NIC power-save mode is a major bottleneck. Simply keeping the radio "awake" reduced transmission delay by up to 60%.
  • TCP is Overkill: For edge-assisted vision, the reliability of TCP comes at too high a cost in time. Sending images as back-to-back UDP bursts was 3x faster than TCP because it avoided the "slow start" of congestion control and the contention caused by return ACKs.
  • Diminishing Returns of Resolution: For large objects, recognition accuracy plateaus quickly as the neural network input size increases. However, for medium-sized objects, increasing the resolution continues to provide benefits much longer.
  • Sharp Trade-offs: There is a "sweet spot" for NN size. Moving from a 128x128 input to 256x256 yields a massive jump in accuracy with a relatively small latency penalty, but further increases show much steeper costs for smaller gains.

Key Results

  • Transmission Speedup: By using UDP and disabling NIC power-save, transmission time was reduced by 65% (from ~15-20ms down to ~2-6ms).
  • Precision Gains: Under real-time constraints (30 fps), the optimized system provided 33% better precision than a "vanilla" setup using standard TCP and default power settings.
  • Frame Rate Boost: For scenarios where lower accuracy is acceptable, the optimized system could achieve up to a 93% higher frame rate than the non-optimized version.
  • Real-time Feasibility: The system successfully maintained satisfactory precision (0.5 AP) even at 40 fps, proving that edge-assisted AR and video analytics are viable with careful tuning.

Limitations and Future Work

  • Network Congestion: The study was conducted over a high-performance 802.11ac link with minimal interference. The authors note that performance might degrade in crowded Wi-Fi environments where packet loss is higher.
  • Battery Impact: Keeping the NIC awake and using high-power GPU offloading has significant implications for mobile battery life, which the paper acknowledges but does not deeply analyze.
  • Single User Focus: The current model focuses on a single client-server pair. Future work could explore how these optimizations scale when dozens of users are competing for the same edge server and wireless bandwidth.

Summary updated

Abstract

We develop an edge-assisted object recognition system with the aim of studying the system-level trade-offs between end-to-end latency and object recognition accuracy. We focus on developing techniques that optimize the transmission delay of the system and demonstrate the effect of image encoding rate and neural network size on these two performance metrics. We explore optimal trade-offs between these metrics by measuring the performance of our real time object recognition application. Our measurements reveal hitherto unknown parameter effects and sharp trade-offs, hence paving the road for optimizing this key service. Finally, we formulate two optimization problems using our measurement-based models and following a Pareto analysis we find that careful tuning of the system operation yields at least 33% better performance for real time conditions, over the standard transmission method.
BibTeX
@inproceedings{galanopoulos2020measurement,
  author = {Apostolos Galanopoulos and Victor Valls and George Iosifidis and Douglas J. Leith},
  title = {{Measurement-driven Analysis of an Edge-Assisted Object Recognition System}},
  booktitle = {ICC},
  year = {2020},
  url = {https://openreview.net/forum?id=nnIH_qvZdJ},
}

Loading related content...

Figure