Abstract
Big Data analytics, the process of organizing and analyzing data to get useful information, is one of the primary uses of cloud services today. Traditionally, collections of data are stored and processed in a single datacenter. As the volume of data grows at a tremendous rate, it is less efficient for only one datacenter to handle such large volumes of data from a performance point of view. Large cloud service providers are deploying datacenters geographically around the world for better performance and availability. A widely used approach for analytics of geo-distributed data is the centralized approach, which aggregates all the raw data from local datacenters to a central datacenter. However, it has been observed that this approach consumes a significant amount of bandwidth, leading to worse performance. A number of mechanisms have been proposed to achieve optimal performance when data analytics are performed over geo-distributed datacenters. In this paper, we present a survey on the representative mechanisms proposed in the literature for wide area analytics. We discuss basic ideas, present proposed architectures and mechanisms, and discuss several examples to illustrate existing work. We point out the limitations of these mechanisms, give comparisons, and conclude with our thoughts on future research directions.
The Bottleneck of Centralization
In the early days of Big Data, the standard assumption was that data resided in a single, massive cluster. Today, global enterprises generate logs, user activity, and sensor data across dozens of geographically dispersed datacenters. The "naive" solution to analyzing this data is the centralized approach: copying all raw data from local sites to a central hub for processing.
As illustrated in the figure above, this model faces three critical challenges:
- Bandwidth Costs: Moving terabytes of raw data over the Wide Area Network (WAN) is prohibitively expensive.
- Network Scarcity: WAN capacity growth lags behind data volume growth, creating a permanent bottleneck.
- Latency: Shipping data across the globe introduces significant delays, making real-time analytics impossible.
Shifting the Paradigm: Distributed Analytics
To overcome these hurdles, a new generation of "Wide Area Analytics" frameworks has emerged. The core philosophy is to move the computation (the "code") to the data, rather than the other way around. By performing initial processing, filtering, and aggregation locally at each datacenter, only the much smaller intermediate results need to be transmitted over the WAN.
This distributed execution can reduce data traffic dramatically—in some scenarios, from 600 GB/day to just 5 GB/day.
Survey of Representative Frameworks
Several frameworks tackle geo-distributed analytics using distinct optimization strategies:
- Graph Partitioning (Pixida): Models data processing as a Directed Acyclic Graph (DAG) and uses "min k-cut" algorithms to minimize data crossing expensive WAN links.
- Caching & Workload Analysis (WANalytics): Caches intermediate results and transmits only the "delta" (differences) between consecutive similar queries.
- Relational Optimization (Geode): Built on Apache Hive, it uses Integer Linear Programming (ILP) to find mathematically optimal execution plans for SQL queries across sites.
- Task & Data Placement (Iridium): Focuses on latency by identifying bottleneck links and proactively moving data to faster sites before queries arrive.
- Job Scheduling (SWAG): Addresses "tail latency" by reordering tasks so that slow datacenters do not delay the entire job completion.
Key Insights and Results
The research highlights several fundamental principles for wide area analytics:
- Data Gravity: Moving small code to large data is almost always superior to moving large data to code.
- Dataflow Forking: If a task's output is needed by multiple sites, it should be sent across the WAN once and replicated locally.
- Predictive Movement: Since many analytics queries are repetitive, observing "query lag" allows systems to pre-position data, effectively "hiding" transfer times.
Specialized frameworks offer significant gains: Pixida can achieve up to a 9x reduction in inter-datacenter traffic compared to standard Spark, while the SWAG algorithm can improve average job completion times by up to 50%.
Limitations and Future Directions
Despite these gains, several challenges remain. Optimization methods like ILP struggle to scale beyond a small number of datacenters. Furthermore, legal constraints like GDPR require "policy-aware" scheduling to ensure data does not leave specific jurisdictions. Future work must also address the instability of WAN bandwidth and the complexity of iterative machine learning workloads across geographically distributed sites.
Summary updated