An Empirical Study of Distributed Deep Learning Training on Edge (Student Abstract)

Christine Mwase, Albert Njoroge Kahira, Zhuo ZouAAAI 2024 · DOI

TL;DR

This paper investigates the feasibility of distributed deep learning (DDL) training on low-cost, low-energy edge devices like Raspberry Pis. The authors found that while DDL is possible, significant challenges remain in optimizing compute, communication, and memory usage across different frameworks, highlighting a need for better software-hardware co-design for edge AI.

What Problem Does This Paper Address?

Deep learning (DL) has revolutionized many fields, but its reliance on powerful, expensive hardware like high-end GPUs and supercomputers creates a significant barrier to entry. This "compute divide" limits access to transformative AI technologies for many researchers and developers. The authors aim to bridge this gap by exploring whether distributed deep learning (DDL) can be effectively performed on low-energy, low-cost edge devices, making AI training more accessible and democratized. They specifically address the challenges of optimizing DL frameworks for these resource-constrained environments, focusing on compute, communication, and memory trade-offs.

Methods

The study employs a multi-faceted approach to evaluate DDL on edge devices:

  • Benchmarking: The LINPACK benchmark was used to verify the setup and establish a baseline performance against theoretical and previous work.
  • Framework Evaluation: The authors trained a ResNet-18 model on the CIFAR-10 dataset using popular DL frameworks (Chainer, PyTorch, and TensorFlow) on Raspberry Pi 4 devices. This allowed them to assess compute, communication, and memory performance of individual devices.
  • Distributed Training: DDL was then implemented and evaluated across multiple Raspberry Pi 4 devices to understand scaling behavior in a cluster setup.
  • Hardware-Level Analysis: Beyond software metrics, the paper delves into hardware-specific performance by analyzing instruction counts, memory accesses, cache behavior, and branch prediction to understand how effectively different frameworks utilize low-level CPU resources.

Datasets

  • CIFAR-10: Used for training and evaluating the ResNet-18 deep learning model.
  • LINPACK Benchmark: Used for initial system verification and performance baselining.

Key Insights and Ideas

  • Edge DDL is Achievable, but Not Seamless: The study demonstrates that it's possible to set up and run DDL on low-cost edge devices, opening doors for more accessible AI training.
  • Frameworks Matter (A Lot): Different DL frameworks exhibit significant performance variations on edge hardware, impacting throughput, memory usage, and efficiency. This highlights the need for framework-specific optimizations for edge deployments.
  • Software-Hardware Co-Design is Crucial: The performance differences are not just about algorithmic efficiency but also how well frameworks leverage the underlying hardware. Optimizing for low-end CPUs in non-HPC settings requires a deeper understanding of hardware capabilities.
  • Current DL Libraries Aren't Edge-Optimized: Existing DL frameworks are largely designed for high-performance computing environments. They often fail to efficiently utilize the specific resources and constraints of low-energy edge devices, leading to suboptimal performance.
  • Scaling is Limited: While some scaling was observed across multiple nodes, it was not linear, indicating bottlenecks that need to be addressed for effective distributed training on edge clusters.

Key Results

  • Performance Variations: Chainer showed a notable advantage in training speed, completing tasks in approximately 47-53% of the time taken by PyTorch and TensorFlow at a batch size of 256. It was also the easiest to install.
  • Memory Consumption: TensorFlow consumed the most memory, while PyTorch used the least, explaining TensorFlow's limitations with higher batch sizes.
  • Scaling: Initial results showed scaling performance across 4 nodes, but this scaling was not linear and can be improved.
  • Hardware Metrics: PyTorch exhibited the highest instruction throughput, likely due to its ability to exploit parallelism and use low-level primitives, effectively leveraging caching and branch prediction. TensorFlow, conversely, had fewer instructions but lower execution rates and a higher L1 cache miss ratio.
  • Throughput: Throughput generally increased with batch size until an optimal point was reached, after which loading fewer batches reduced overhead. Frameworks were limited to batch sizes of 512 and 1024 due to memory constraints.

Limitations and Future Work

  • Limited Hardware: The study was conducted on Raspberry Pi 4 devices with 4GB of LPDDR4 SDRAM. Further evaluation on a wider range of edge devices with varying specifications is needed.
  • Model Complexity: Only ResNet-18 on CIFAR-10 was evaluated. More complex models and datasets would provide a more comprehensive understanding of DDL capabilities on edge.
  • Framework Optimization: The authors acknowledge that current DL libraries are not optimized for low-end CPUs in non-HPC settings. Future work should focus on software-hardware co-design and optimization for these environments.
  • Communication Bottlenecks: Addressing communication bottlenecks in distributed edge training is a key area for future research.
  • Tooling: There is a lack of dedicated tools and support for DL training on low-energy edge devices, similar to the TinyML initiative. Developing such tools is crucial for broader adoption.

Summary updated

BibTeX
@inproceedings{mwase2024empirical,
  author = {Christine Mwase and Albert Njoroge Kahira and Zhuo Zou},
  title = {{An Empirical Study of Distributed Deep Learning Training on Edge (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30485},
}
Figure