Facilitating Near Real Time Analytics on the Edge

Ivona BrandicMECO 2019 · DOI

Abstract

As the Internet of Things (IoT) transitions from centralized cloud models to decentralized edge architectures, managing the trade-off between low-latency requirements and energy efficiency becomes critical. This paper outlines a research roadmap for facilitating near real-time analytics across distributed micro data centers, highlighting techniques such as dependency mining, DAG offloading, and control-theoretic resource management.

Introduction

The traditional cloud computing model is increasingly strained by the volume and velocity of data generated by modern IoT ecosystems. Applications such as autonomous vehicles, smart city infrastructure, and remote surgery require near-instantaneous processing that centralized data centers cannot provide due to inherent network latency. Consequently, the industry is moving toward "Edge Computing," where data processing occurs in proximity to the source.

Ivona Brandic, keynote speaker at MECO 2019.

However, the edge is not a monolithic entity but a fragmented landscape of "micro data centers." This shift introduces a primary conflict: deploying large-scale, high-stakes applications across these micro data centers while guaranteeing high Quality of Service (QoS)—specifically low latency—and minimizing the energy footprint of the hardware. Managing this trade-off at scale is the central hurdle for the next generation of digital infrastructure.

Methodological Approaches to Edge Complexity

To address the complexities of the edge, several specialized techniques have been developed to enhance resilience and efficiency:

  • Dependency Mining: Identifying inter-service dependencies to improve the overall resilience of edge-hosted applications against failures.
  • DAG-Based Offloading: Utilizing Directed Acyclic Graphs (DAGs) to model computations allows systems to determine the optimal "first hop" for mobile offloading, ensuring tasks are processed at the most geographically and computationally appropriate location.
  • Control-Theoretic Resource Management: Hybrid cloud controllers employing control theory can manage "memory elasticity," enabling the dynamic adjustment of resources in response to real-time demand fluctuations.
  • Geotemporal Modeling: Developing controllers that incorporate both temporal and spatial (geotemporal) inputs allows for more effective optimization of resource distribution across cloud and edge tiers.

Key Insights: The QoS-Energy Paradox

A fundamental insight of this research is the "QoS-Energy Paradox." In centralized cloud environments, latency is often mitigated by over-provisioning resources. At the edge, however, resources are finite and power consumption is a critical constraint. Energy efficiency is no longer merely a sustainability goal but a hard technical requirement for the operation of resource-constrained edge nodes.

Furthermore, the challenge extends beyond initial deployment to runtime operations. Because edge environments are inherently dynamic—with moving devices and fluctuating signal strengths—systems must be capable of autonomous reconfiguration to maintain near real-time performance. This shift from massive data centers to micro data centers requires a fundamental rethink of network topology and management.

Conclusion and Future Outlook

While current research demonstrates the feasibility of performance-based pricing and vertical memory elasticity, the increasing complexity of multi-data center environments continues to challenge existing management tools. The future of edge computing lies in the development of autonomous, self-governing ICT infrastructures. Projects like FoSII aim to create systems that can self-heal and self-optimize, providing the scalability necessary to manage the projected growth of the IoT landscape without constant human intervention.

Summary updated

BibTeX
@inproceedings{brandic2019facilitating,
  author = {Ivona Brandic},
  title = {{Facilitating Near Real Time Analytics on the Edge}},
  booktitle = {MECO},
  year = {2019},
  doi = {10.1109/meco.2019.8760130},
}
Figure