Supporting data aspects in pig latin

Curtis E. Dyreson, Omar U. Florez, Akshay Thakre, Vishal SharmaAOSD 2013 · DOI

TL;DR

This paper introduces a novel approach to integrate "data aspects" – cross-cutting concerns like privacy, security, and lineage – into Pig Latin, a popular dataflow language for cloud-based big data analysis. By adapting Aspect-Oriented Programming (AOP) principles, they enable Pig Latin programs to automatically enforce these data semantics without requiring manual code modifications, making big data analysis more robust and secure.

What Problem Does This Paper Address?

Analyzing massive datasets in the cloud, often using languages like Pig Latin, presents a significant challenge: managing "cross-cutting concerns" related to the data itself. These concerns, such as data privacy, security policies, versioning, and provenance (tracking data origins), are universal and can affect many different datasets. Traditional data management systems and languages like Pig Latin lack built-in mechanisms to handle these concerns uniformly. Developers are forced to resort to ad-hoc solutions, leading to complex, error-prone, and difficult-to-maintain code. This paper aims to bridge this gap by providing a systematic framework to weave these data aspects directly into Pig Latin programs.

Methods

The core contribution of this paper is the adaptation of Aspect-Oriented Programming (AOP) to Pig Latin. AOP allows developers to modularize and inject cross-cutting concerns into existing code without altering the core logic. In this context, the authors define "data aspects" as metadata that "tags" data with information related to these concerns.

They introduce a "data weaver" that transforms a standard Pig Latin program into an "aspected" version. This weaver uses Pig Latin templates or patterns to rewrite the original program. For each Pig Latin transformation (like FILTER, JOIN, GROUP), they define corresponding "aspected" versions (FILTERAO, JOINAO, GROUPAO). These aspected transformations ensure that the associated advice (metadata) is processed alongside the data at each step.

Key techniques include:

  • Data Tagging: Data is augmented with metadata (advice) representing aspects like security or lineage.
  • Data Cuts: A mechanism to link specific data instances to their corresponding advice.
  • Aspect Weaving: A process that modifies the Pig Latin program to incorporate aspect-specific operations before, during, or after the execution of original transformations.
  • Advice In-lining: An optimization technique to improve performance by merging advice with data early in the query execution.
  • Meta-metadata: The framework supports recursive levels of advice, allowing metadata to be advised by further metadata.

Datasets

The paper uses a running example based on a hypothetical "Magazine.com" subscriber database to illustrate its concepts. This example includes subscriber information (name, city, amount, ID) and personal information (email). No new benchmarks or large-scale datasets were introduced; the focus is on the methodology and its application to a conceptual dataset.

Key Insights and Ideas

  • Data as a First-Class Aspect Target: The paper convincingly argues that data, like code, has inherent cross-cutting concerns that need systematic management, moving beyond simple descriptive metadata.
  • AOP as a Unifying Framework: Applying AOP principles provides a powerful and elegant way to address diverse data concerns (privacy, lineage, temporal, security) within a single, coherent framework.
  • Pig Latin's Suitability for AOP: The dataflow nature of Pig Latin and its statement-based transformations make it a good candidate for aspect-oriented extensions, allowing for modular weaving of advice.
  • Meta-metadata for Complex Concerns: The ability to layer advice (meta-metadata) is crucial for modeling sophisticated scenarios, such as versioned security, where the definition of security itself might change over time.
  • Performance Trade-offs: While aspect-oriented processing adds overhead, the paper demonstrates that optimizations like advice in-lining can mitigate performance impacts, and the benefits of enforced semantics often outweigh the cost.

Key Results

The paper demonstrates the feasibility of aspect-oriented Pig Latin by defining aspected versions of core Pig Latin transformations. They provide concrete examples of how aspects like security and lineage are woven into queries. Experimental results (Table 13) show that while aspect-oriented operations incur additional costs (e.g., FILTERAO is ~24x slower in the worst case for 5 million tuples), the overhead is manageable, especially with optimizations like advice in-lining. The authors highlight that the added cost is for enforcing critical data semantics that are otherwise absent.

Limitations and Future Work

  • UDF Integration: User-Defined Functions (UDFs) in Pig Latin are currently not aspect-aware. Developers must rewrite UDFs to handle aspected data, which is a significant burden. Future work aims to address this by either partitioning data for UDFs or re-engineering UDF languages (like Java) to support data aspects.
  • Data Management Role: Pig Latin itself doesn't have a data management role. Future work includes investigating mechanisms for tagging data with advice before it enters the Pig Latin processing pipeline.
  • Optimization: Further research into optimization rules for aspect-oriented Pig Latin is planned, particularly exploring keeping relations joined with their advice throughout evaluation.
  • Complexity: The paper acknowledges that aspect-oriented data processing increases complexity and cost, and further analysis and optimization are needed.
  • Program Aspects: While the paper focuses on data aspects, it briefly mentions "program aspects" (constraints on relations evaluated by a program) and suggests this as a future direction.

Summary updated

BibTeX
@inproceedings{dyreson2013supporting,
  author = {Curtis E. Dyreson and Omar U. Florez and Akshay Thakre and Vishal Sharma},
  title = {{Supporting data aspects in pig latin}},
  booktitle = {AOSD},
  year = {2013},
  doi = {10.1145/2451436.2451439},
}

Loading related content...

Figure