Data Analytics for Fog Computing by Distributed Online Learning with Asynchronous Update

Guangxia Li, Peilin Zhao, Xiao Lu, Jia Liu, Yulong Shen

Abstract

Fog computing extends the cloud computing paradigm by allocating substantial portions of computations and services towards the edge of a network, making it ideal for large-scale, geo-distributed, and data-intensive applications. To address the need for smart data analytic tools in this environment, we propose a system where data streams from distributed sources are processed locally, with only distilled information sent to a central node. This center extracts knowledge and shares it back to subordinates to boost performance. We implement a distributed machine learning algorithm using an online learning approach, enhanced by an asynchronous update strategy for system robustness. Experimental results show that our method achieves classification accuracy comparable to centralized models while significantly improving efficiency and scalability.

Introduction: The Latency and Privacy Challenge

While cloud computing offers immense processing power, it faces significant hurdles in the Internet of Things (IoT) era: latency and privacy. Transmitting raw sensor data to distant data centers is slow and exposes sensitive information.

The three-layer architecture of fog computing, connecting smart end-devices to the cloud via intermediate fog nodes.

Fog computing addresses these issues by moving computation to nodes closer to the data source, such as smart routers or local servers. However, standard machine learning tools are often ill-suited for the distributed, real-time, and sometimes unreliable nature of edge networks. Our work fills this gap by enabling massive data streams to be processed locally while still benefiting from shared global knowledge.

Proposed Framework: Distributed Online Multitask Learning (DOML)

Our system utilizes a Master/Worker architecture built on three core pillars:

  1. Online Multitask Learning: Unlike traditional batch learning, online learning processes data samples one at a time, which is essential for continuous streams. The "multitask" aspect recognizes that while different edge devices (Workers) may see unique data distributions, their underlying tasks are related.
  2. Asynchronous Updates: To handle real-world network variability where some nodes are slower than others, the Master node updates the global model as soon as it receives a gradient from any worker, rather than waiting for all. A "delayed gradient" strategy ensures mathematical convergence despite these timing gaps.
  3. Communication Efficiency: To minimize network traffic, Workers transmit a "distilled" gradient vector ($\bar{g}$). The Master then uses a predefined interaction matrix ($A$) to reconstruct the full gradient, shifting the computational burden away from the bandwidth-constrained edge.
The Master/Worker communication flow, showing data ingestion at workers and the exchange of distilled gradients and model updates.

Experimental Evaluation

We evaluated the DOML framework using a synthetic dataset designed to mimic complex, non-linear classification tasks across 64 distinct data sources. These tasks were related via "random walks" in a parameter space, simulating how different sensors in a network might have unique but related perspectives. To handle non-linearity, original 2D data was mapped into a 9D feature space.

Key Results

  • High Accuracy: The DOML algorithm achieved a cumulative error rate of 24.22%, nearly identical to a centralized multitask model (24.67%) and significantly better than a standard "one-size-fits-all" online learner (28.96%).
  • Scalability and Efficiency: By digesting data locally, the system scales effectively as more Workers are added. In tests, the system processed hundreds of thousands of samples in seconds on standard consumer hardware.
  • Robustness: The mathematical support for delayed gradients allows the system to remain functional even when edge devices experience poor connectivity.

Conclusion and Future Directions

The DOML framework demonstrates that distributed edge networks can achieve high-performance data analytics without the overhead of centralized cloud processing. Future work will focus on testing the system with massive real-world datasets and developing methods to dynamically learn the task interaction matrix ($A$) as data flows through the network.

Summary updated

BibTeX
@inproceedings{li2019data,
  author = {Guangxia Li and Peilin Zhao and Xiao Lu and Jia Liu and Yulong Shen},
  title = {{Data Analytics for Fog Computing by Distributed Online Learning with Asynchronous Update}},
  booktitle = {ICC},
  year = {2019},
  url = {https://openreview.net/forum?id=fW15_wvML1u},
}
Figure