Probabilistic Time Series Forecasting for Adaptive Monitoring in Edge Computing Environments

Dominik Scheinert, Babak Sistani Zadeh Aghdam, Soeren Becker, Odej Kao, Lauritz Thamsen

TL;DR

This paper introduces a novel adaptive monitoring system for resource-constrained edge environments that leverages probabilistic time series forecasting in the cloud. By predicting future metric values and quantifying their uncertainty, the system intelligently decides when to fetch actual data from edge devices, significantly reducing network traffic while maintaining accurate monitoring.

What Problem Does This Paper Address?

The increasing shift of computation to the "edge" of networks – closer to data sources like autonomous vehicles or IoT devices – presents a significant challenge for monitoring critical infrastructure. These edge environments are often resource-constrained (limited processing power, battery, and network bandwidth). Traditional monitoring methods, which involve constantly transmitting data or adapting sampling directly on edge devices, are inefficient. Direct adaptation on edge devices further strains their limited resources and restricts the complexity of models that can be employed. This paper tackles the problem of reducing the network overhead associated with monitoring these critical edge infrastructures without compromising the accuracy of the collected data or requiring modifications to the edge devices themselves.

Methods

The core of the proposed solution is a cloud-located adaptive monitoring system that utilizes probabilistic time series forecasting. Instead of relying on edge devices for adaptive logic, the system operates in the cloud.

Here's how it works:

  1. Data Acquisition: The system initially collects data from edge devices at a high frequency to establish a baseline.
  2. Probabilistic Forecasting: A probabilistic time series forecasting model (specifically, using quantile regression) is trained on this historical data. This model doesn't just predict future values but also provides a measure of uncertainty associated with those predictions.
  3. Uncertainty Quantification: The system quantifies the uncertainty of the forecasted metrics. This is done by calculating the standard deviation of the forecasted samples across different quantiles.
  4. Conditional Sampling: Based on the quantified uncertainty, the system makes an intelligent decision:
  5. If the model is confident (low uncertainty), it relies on its forecasted values, thus avoiding fetching actual data from the edge.
  6. If the model is uncertain (high uncertainty), it triggers fetching actual, live metric data from the edge device. This ensures that critical deviations or anomalies are not missed.
  7. Model Retraining: The system periodically retrains its forecasting model to adapt to evolving data streams and prevent model drift.

This approach shifts the computational burden of adaptive monitoring to the cloud, where resources are more abundant, and avoids modifying the resource-constrained edge devices.

Datasets

The paper uses a publicly available streaming dataset for evaluation. The data acquisition process for the experiment involved simulating a workload on a host machine using 10 Python services that consume messages from an external RabbitMQ queue. Metrics from the host machine were gathered via Prometheus queries. The message rate was controlled by the IoT Vehicles experiment dataset [28], which reports the amount of moving cars at 1-second intervals. This dataset was used to publish messages to the queue, thereby simulating varying patterns and generating relevant metrics.

Key Insights and Ideas

  • Cloud-centric adaptive monitoring is viable: Shifting the intelligence for adaptive monitoring to the cloud, rather than the edge, can effectively reduce resource strain on edge devices.
  • Probabilistic forecasts are key to intelligent sampling: The ability to quantify prediction uncertainty is crucial. It allows the system to distinguish between reliable predictions and situations where actual data is necessary, leading to more informed sampling decisions.
  • Contextualized adaptation: The system's adaptive sampling is not a one-size-fits-all approach. It considers the inherent characteristics of individual metrics, leading to more nuanced and efficient data reduction.
  • No edge device modification needed: A significant advantage is that the proposed system does not require any changes to the edge devices themselves, making it highly practical for deployment in existing infrastructures.

Key Results

The paper presents a comparison between their proposed AM-PF (Adaptive Monitoring with Probabilistic Forecasting) and a related method, AM-DR (Adaptive Monitoring - Data Reduction) [13].

  • Data Reduction: Both methods demonstrate significant reductions in transmitted data as thresholds increase. AM-PF shows a more metric-specific adaptation of sampling frequencies.
  • Prediction Accuracy: While higher thresholds generally lead to lower accuracy for both methods, AM-PF's accuracy is described as "more volatile in the beginning and tend to converge toward a common value," whereas AM-DR's SMAPE values "steadily increasing and tend to fan out over time."
  • Threshold Impact: The choice of threshold significantly impacts the trade-off between data transmission reduction and prediction accuracy. A more conservative threshold (e.g., 0.0225 for AM-PF) leads to more frequent fetching of real metrics and higher accuracy, while a more opportunistic threshold (e.g., 0.0475) reduces bandwidth but can lead to less accurate results.

Limitations and Future Work

  • Assumptions: The approach assumes that target edge devices expose accessible metric endpoints. While a reasonable assumption, some edge nodes might require adaptations if they primarily use push-based messaging.
  • Forecasting Horizon: The choice of the forecasting horizon is important and should allow for flexible reaction to changes in the data stream.
  • Resource Availability: The system assumes sufficient resources are available at the sink node (cloud) for model training.
  • Future Work: The authors plan to leverage their findings for research on carbon-aware computing in edge environments, aiming to optimize resource usage and energy efficiency.

Summary updated

Abstract

With increasingly more computation being shifted to the edge of the network, monitoring of critical infrastructures, such as intermediate processing nodes in autonomous driving, is further complicated due to the typically resource-constrained environments. In order to reduce the resource overhead on the network link imposed by monitoring, various methods have been discussed that either follow a filtering approach for data-emitting devices or conduct dynamic sampling based on employed prediction models. Still, existing methods are mainly requiring adaptive monitoring on edge devices, which demands device reconfigurations, utilizes additional resources, and limits the sophistication of employed models.In this paper, we propose a sampling-based and cloud-located approach that internally utilizes probabilistic forecasts and hence provides means of quantifying model uncertainties, which can be used for contextualized adaptations of sampling frequencies and consequently relieves constrained network resources. We evaluate our prototype implementation for the monitoring pipeline on a publicly available streaming dataset and demonstrate its positive impact on resource efficiency in a method comparison.
BibTeX
@inproceedings{scheinert2022probabilistic,
  author = {Dominik Scheinert and Babak Sistani Zadeh Aghdam and Soeren Becker and Odej Kao and Lauritz Thamsen},
  title = {{Probabilistic Time Series Forecasting for Adaptive Monitoring in Edge Computing Environments}},
  booktitle = {IEEE Big Data},
  year = {2022},
  url = {https://openreview.net/forum?id=6QcXs2Osse},
}
Figure