Guardians of the Deep Fog

Ashkan Yousefpour, Siddartha Devic, Brian Nguyen, Aboudy Kreidieh, Alan Liao, Alexandre M. Bayen, Jason P. Jue

TL;DR

This paper introduces deepFogGuard, a novel method to make deep neural network (DNN) inference resilient to physical node failures in distributed edge-fog-cloud systems. By incorporating "skip hyperconnections," deepFogGuard ensures that data can still flow through the network even when some nodes fail, significantly improving inference accuracy compared to standard approaches.

What Problem Does This Paper Address?

The increasing prevalence of Internet of Things (IoT) devices generates vast amounts of data that can be effectively analyzed using Deep Neural Networks (DNNs). To handle the computational demands and latency requirements, DNNs are often distributed across edge, fog, and cloud nodes. However, this distribution introduces a critical vulnerability: if any physical node in the chain fails (due to power outages, cable cuts, or other issues), the DNN units on that node also fail, potentially leading to unpredictable and poor inference performance. This is particularly problematic for critical applications where consistent performance is paramount. The paper aims to address this challenge by developing a method to ensure DNN inference remains robust even when physical nodes in the distributed system fail.

Methods

The core contribution of deepFogGuard lies in its architecture augmentation strategy. Inspired by residual connections in DNNs, the authors introduce skip hyperconnections. While standard DNNs use residual connections to skip layers within a single node, skip hyperconnections in deepFogGuard are designed to skip physical nodes in a distributed DNN.

Here's how it works:

  • Distributed DNNs: A DNN is partitioned and its layers are spread across multiple physical nodes (edge, fog, cloud).
  • Hyperconnections: These are the communication links between physical nodes that transfer intermediate DNN outputs.
  • Skip Hyperconnections: These are special hyperconnections that bypass one or more intermediate physical nodes, creating alternative data pathways.
  • Expansion Layer & Add Operation: To handle data from multiple hyperconnections (including skips), each physical node that receives data via hyperconnections has an "expansion layer." An "Add operation" combines the outputs from incoming hyperconnections before they are fed into the expansion layer. This operation is designed to ignore null vectors (representing failed nodes).
  • Resiliency: When a physical node fails, the data flow can be rerouted through the skip hyperconnections, preventing a complete breakdown of the inference process. This is a form of "passive resiliency," meaning it functions without requiring immediate re-training or active intervention.

The training process for deepFogGuard can be performed in a simulated environment where the DNN is distributed, allowing the model to learn to leverage these skip hyperconnections for improved robustness.

Datasets

The paper evaluates deepFogGuard using two existing datasets:

  • Health Activity Classification ("Health"): This dataset, derived from the UCI MHealth dataset, uses sensor data (acceleration, orientation, ECG) from different body parts to classify 12 human activities. It's used to test a vertically distributed DNN.
  • Multi-Camera Object Classification ("Camera"): This dataset, based on the multi-view object detection dataset, provides videos from six different viewpoints, with bounding boxes for pedestrians, cars, and buses. It's used to test a DNN distributed both vertically and horizontally.

Key Insights and Ideas

  • Skip Hyperconnections as "Guardians": The name deepFogGuard and the concept of skip hyperconnections highlight their role in protecting the data flow. They act as a safeguard, ensuring that even if a direct path is broken by a node failure, information can still reach its destination.
  • Passive Resiliency is Key: The focus on passive resiliency means the system can handle failures without complex real-time adaptation or re-training, which is crucial for time-sensitive IoT applications.
  • Simulated Training Suffices: The ability to train the resilient DNN in a simulated distributed environment simplifies the training process, as it doesn't require actual physical node failures during training.
  • Vertical vs. Horizontal Distribution Matters: The experiments show that the benefits of deepFogGuard are more pronounced in highly distributed architectures (both vertical and horizontal), where the potential for single-node failures to disrupt the entire pipeline is higher.

Key Results

The experiments demonstrate that deepFogGuard significantly improves inference accuracy in the presence of physical node failures compared to a "Vanilla" (non-resilient) distributed DNN.

  • Health Activity Classification: In the "Hazardous" reliability setting (frequent failures), deepFogGuard achieved an average accuracy increase of nearly 16% over Vanilla. In the "Poor" setting, the improvement was around 6%.
  • Multi-Camera Object Classification: Even with a more robust Vanilla architecture, deepFogGuard consistently outperformed it across all reliability settings. The largest gains were again observed in the "Hazardous" setting.
  • General Trend: In both experiments, deepFogGuard showed improved accuracy across all reliability settings, with the most substantial benefits appearing when failure probabilities were higher.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Bandwidth Consumption: deepFogGuard can consume additional bandwidth when there are no failures, as skip hyperconnections are always active.
  • Resource Overhead: Maintaining multiple TCP connections and checking node statuses incurs some resource overhead.
  • Extending to Other Architectures: Future work could explore extending deepFogGuard to other DNN architectures like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).
  • Integration with Robustness Techniques: Investigating the combination of deepFogGuard with regularization techniques like dropout, or methods that implicitly increase robustness, is a promising avenue.
  • Adaptive Hyperconnection Weights: The paper suggests that adjusting the weights of remaining hyperconnections after a node failure could further account for changes in input scale and improve performance.

Summary updated

Abstract

Partitioning and distributing deep neural networks (DNNs) over physical nodes such as edge, fog, or cloud nodes, could enhance sensor fusion, and reduce bandwidth and inference latency. However, when a DNN is distributed over physical nodes, failure of the physical nodes causes the failure of the DNN units that are placed on these nodes. The performance of the inference task will be unpredictable, and most likely, poor, if the distributed DNN is not specifically designed and properly trained for failures. Motivated by this, we introduce deepFogGuard, a DNN architecture augmentation scheme for making the distributed DNN inference task failure-resilient. To articulate deepFogGuard, we introduce the elements and a model for the resiliency of distributed DNN inference. Inspired by the concept of residual connections in DNNs, we introduce skip hyperconnections in distributed DNNs, which are the basis of deepFogGuard's design to provide resiliency. Next, our extensive experiments using two existing datasets for the sensing and vision applications confirm the ability of deepFogGuard to provide resiliency for distributed DNNs in edge-cloud networks.
BibTeX
@misc{yousefpour2019guardians,
  author = {Ashkan Yousefpour and Siddartha Devic and Brian Nguyen and Aboudy Kreidieh and Alan Liao and Alexandre M. Bayen and Jason P. Jue},
  title = {{Guardians of the Deep Fog}},
  year = {2019},
  eprint = {1909.00995},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.00995},
}
Figure