Characterizing Perception Deep Learning Algorithms and Applications for Vehicular Edge Computing

Wang Feng, Sihai Tang, Shengze Wang, Ying He, Donger Chen, Qing Yang, Song FuAlgorithms 2025 · DOI

TL;DR

This paper investigates the critical, yet often overlooked, impact of memory contention on deep learning perception workloads in vehicular edge computing. By profiling common perception models like YOLO and Faster R-CNN, the authors reveal how memory limitations can drastically slow down specific network layers, offering a new framework for understanding and predicting performance on resource-constrained edge devices.

What Problem Does This Paper Address?

Autonomous vehicles (AVs) rely heavily on perception systems, powered by deep learning, to make critical driving decisions. These systems generate massive amounts of data that need to be processed efficiently, often at the edge of the network (vehicular edge computing) to minimize latency. While much research focuses on CPU and GPU performance, the limited memory resources on edge devices can become a significant bottleneck, leading to performance degradation or even system failure. This paper addresses the gap in understanding how memory contention specifically impacts the performance of perception deep learning models on these resource-constrained edge platforms.

Methods

The authors employ a profiling-based approach to characterize the behavior of deep learning perception workloads under varying memory conditions. They focus on two prominent object detection models: single-stage YOLO and two-stage Faster R-CNN, implemented in PyTorch. The methodology involves:

  • Testbed Setup: Experiments are conducted on both a high-end laptop and a more representative low-end edge device (Nvidia Jetson Xavier NX) to capture a range of hardware capabilities.
  • Workload Simulation: Standard AV-oriented image datasets (subsets of KITTI and COCO) are used with varying input resolutions to simulate realistic perception tasks.
  • Memory Contention Scenarios: Three scenarios are defined: no memory contention, moderate contention (heavy workloads), and extreme lack of memory. This is achieved by artificially reducing available RAM.
  • Profiling: Inference latency, peak and average memory consumption, CPU frequency, and GPU utilization are meticulously monitored.
  • Layer-wise Analysis: The performance of individual layers within the DNNs is analyzed to pinpoint which components are most affected by memory constraints.
  • Model Development: Mathematical models are derived to characterize and predict workload behavior based on CPU power, memory availability, and input size.

Datasets

The paper utilizes subsets of the following standard datasets for object detection tasks:

  • KITTI: A widely used dataset for autonomous driving research, providing sensor data such as images, LiDAR, and GPS.
  • COCO (Common Objects in Context): A large-scale dataset for object detection, segmentation, and captioning, often used to train and evaluate general-purpose object recognition models.

The authors note that these publicly available datasets represent a small fraction of real-world conditions and suggest the need for more diverse datasets for broader applicability.

Key Insights and Ideas

  • Memory is a Critical Bottleneck: Contrary to the common focus on CPU/GPU, memory contention is shown to be a primary driver of performance degradation in perception workloads on edge devices.
  • Layer-Specific Sensitivity: Not all layers are equally affected. Convolutional, activation, routing, and shortcut layers are identified as particularly vulnerable to memory constraints, experiencing disproportionately large increases in execution time.
  • Dramatic Latency Spikes: Certain layers, like convolutional layers in YOLO, can see computation time increases exceeding 2849%, and activation layers in Faster R-CNN can increase by over 1173.34% under memory contention.
  • Beyond CPU/GPU: Traditional scheduling approaches that only consider CPU and GPU are insufficient for optimizing performance on edge devices due to the significant impact of memory.
  • Predictive Modeling is Possible: The authors demonstrate that workload behavior can be modeled and predicted based on edge device parameters (CPU, memory) and task characteristics (input size), enabling better resource management.
  • Oversubscription Feasibility: Despite memory contention, the studied algorithms (YOLO, Faster R-CNN) can still complete tasks, suggesting that resource oversubscription might be a viable strategy for non-time-sensitive applications.

Key Results

  • Layer Performance Degradation:
    • YOLO: Convolutional layer 6 experienced a 2849% increase in execution time, and layer 1 saw a 2665% increase under memory contention. Routing layer 9 showed a 1053% increase.
  • Faster R-CNN: ReLU activation layer 3 saw a 1173.34% increase, and convolutional layer 7 experienced a 2339.77% increase.
  • Mathematical Models: The paper presents derived equations (e.g., Equations 2-5) that model the relationship between inference time, CPU power, and available memory for both YOLO and Faster R-CNN under different scenarios.
  • Generalization: The findings suggest that the developed characterization models can be generalized to some extent across different edge platforms and ML algorithms.
  • OOM Killer Anomaly: An interesting observation was that oversubscribed workloads triggered the Out-Of-Memory (OOM) killer on Nvidia Jetson but not on other platforms with similar Linux kernels, suggesting platform-specific behaviors.

Limitations and Future Work

  • Limited AV Dynamics: The study focuses on memory contention and latency but does not explicitly account for dynamic vehicular mobility, network conditions (e.g., handoffs), or their impact on resource allocation.
  • Dataset Limitations: The use of publicly available datasets means the models and findings might not fully represent real-world conditions, necessitating more diverse datasets.
  • Layer Anomaly Investigation: The disproportionate latency spikes in routing and shortcut layers, despite their low computational footprint, were hypothesized to stem from architectural design or memory bottlenecks but were not fully isolated due to technical complexities. Deeper hardware/machine-level studies are suggested.
  • Encryption/Obfuscation Impact: The current work does not quantitatively evaluate how privacy-enhancing techniques like encryption or obfuscation might impact memory contention.
  • Future Work:
    • Develop accurate and efficient profiling tools and methodologies for edge devices.
  • Establish standardized benchmarks and metrics for generalized use cases.
  • Investigate the impact of privacy-preserving techniques on memory contention.
  • Explore distributed systems and edge-to-cloud architectures to mitigate contention.
  • Further research into the latency anomalies of specific layers and explore parallelizable architectures like transformers.
  • Develop more robust predictive models that can adapt to individual edge nodes through self-updating loops, requiring larger datasets.

Summary updated

Abstract

Vehicular edge computing relies on the computational capabilities of interconnected edge devices to manage incoming requests from vehicles. This offloading process enhances the speed and efficiency of data handling, ultimately boosting the safety, performance, and reliability of connected vehicles. While previous studies have concentrated on processor characteristics, they often overlook the significance of the connecting components. Limited memory and storage resources on edge devices pose challenges, particularly in the context of deep learning, where these limitations can significantly affect performance. The impact of memory contention has not been thoroughly explored, especially regarding perception-based tasks. In our analysis, we identified three distinct behaviors of memory contention, each interacting differently with other resources. Additionally, our investigation of Deep Neural Network (DNN) layers revealed that certain convolutional layers experienced computation time increases exceeding 2849%, while activation layers showed a rise of 1173.34%. Through our characterization efforts, we can model workload behavior on edge devices according to their configuration and the demands of the tasks. This allows us to quantify the effects of memory contention. To our knowledge, this study is the first to characterize the influence of memory on vehicular edge computational workloads, with a strong emphasis on memory dynamics and DNN layers.
BibTeX
@inproceedings{feng2025characterizing,
  author = {Wang Feng and Sihai Tang and Shengze Wang and Ying He and Donger Chen and Qing Yang and Song Fu},
  title = {{Characterizing Perception Deep Learning Algorithms and Applications for Vehicular Edge Computing}},
  booktitle = {Algorithms},
  year = {2025},
  doi = {10.3390/a18010031},
}
Figure