Global Optimization through Deferred Execution

38 concepts29 papers1 proposal

Imagine a courier tasked with delivering packages across a sprawling city. If the courier sprints out the door the exact moment the first package arrives at the depot, they are practicing eager execution. While this minimizes the latency for that single package, it is globally blind. The courier will likely backtrack across town when the second and third packages arrive minutes later, wasting immense time and energy.

Now imagine an alternative: the courier waits at the depot, collecting requests over a window of time to build a complete manifest. With this holistic view, they can plan the most efficient route, group deliveries by neighborhood, and completely bypass redundant trips.

In modern computing—spanning relational databases, data science libraries, and deep learning frameworks—this strategy is known as deferred execution (or lazy evaluation). By intentionally delaying computation and data materialization until a concrete result is strictly required, systems can shift their optimization boundaries from isolated, local statements to entire pipelines of work.


Intercepting Intent: Building the Blueprint

To optimize a computation globally, a system must first capture the user's intent without actually executing the underlying operations. This is achieved by transparently intercepting imperative API calls and recording them as nodes in a symbolic intermediate representation (IR), directed acyclic graph (DAG), or logical query tree.

In data science, users heavily rely on imperative dataframe APIs like Pandas, which traditionally execute operations eagerly and frequently hit memory bottlenecks. To solve this, Emani developed PyFroid, a framework that intercepts standard Pandas API calls to construct a symbolic query plan. Instead of immediately manipulating data in memory, PyFroid defers execution, allowing operations to be pushed down to an optimized relational engine like DuckDB. Similarly, Jindal introduces Magpie, which intercepts dataframe operations to build a global logical query tree, enabling seamless cross-operation optimizations before offloading the work to high-performance cloud backends.

This interception strategy is equally vital in distributed array computing and machine learning. Huang et al. demonstrate in Spartan: A Distributed Array Framework with Smart Tiling how lazy evaluation captures user programs as expression graphs to optimize data partitioning before distributed execution begins. In deep learning, early frameworks like TensorFlow (Abadi et al.) and Theano (Al-Rfou et al.) decoupled graph definition from execution to enable hardware-specific kernel tuning. Even within dynamic neural network toolkits like DyNet, Neubig et al. utilize deferred execution to separate graph construction from numerical calculations, minimizing immediate overhead during complex, variable-structured model training.


What Deferral Unlocks: The Global Optimization Window

Why go through the trouble of building a symbolic graph instead of executing code immediately? The answer lies in the powerful optimizations that a global view unlocks.

1. Structural Pruning and Elimination

When a system can see the entire pipeline, it can identify and eliminate computations or data paths that do not contribute to the final output. For instance, Al-Rfou et al. leverage the symbolic execution engine of Theano to automatically identify and ignore branches of a computation graph that are irrelevant to the requested output. In data pipelines, Jindal demonstrates that a deferred query tree allows for aggressive column pruning and early row filtering—optimizations that are impossible if each line of code is executed eagerly.

2. Cross-Boundary Operator Fusion

In an eager system, the output of one function must be fully materialized in memory before being passed to the next. This creates a massive "data movement bottleneck." By deferring execution, compilers can merge adjacent operations. Palkar et al. address this with Weld, a runtime that compiles functional IR fragments from disparate libraries (like NumPy and Spark) into a single DAG. This global view allows Weld to perform loop fusion across library boundaries that would otherwise be opaque to one another. Similarly, Zhou et al. show that Jittor-GAN leverages deferred execution to identify operator fusion opportunities, significantly accelerating the training of generative adversarial networks.

3. On-the-Fly Batching and Scheduling

Deferral allows systems to accumulate independent operations and execute them collectively to maximize hardware throughput. In On-the-fly Operation Batching in Dynamic Computation Graphs, Neubig et al. construct a symbolic representation of an entire minibatch before execution, allowing the framework to automatically group instance-level operations into high-performance batched operations on the fly.

For spatial data processing, Gieseke et al. employ a lazy evaluation strategy in Buffer k-d Trees, delaying nearest-neighbor query execution until enough queries accumulate for a specific spatial partition, prioritizing total GPU throughput over individual query latency. In database applications, Cheung et al. introduce "extended lazy evaluation" via tools like SLOTH. Instead of making immediate network round trips to a database, SLOTH returns placeholders (thunks) to the application server, collecting deferred requests into a query store and executing them as a single optimized batch only when a concrete value is required.


Lazy Materialization: Minimizing Data Movement

A critical extension of deferred execution is lazy materialization—the practice of delaying the loading, parsing, or formatting of data values until the last possible moment.

Eagerly loading entire rows or parsing complex data structures into memory creates severe overhead, especially if subsequent operations (like filters or joins) discard most of the data. As Ozcan et al. noted in the context of multidatabases, delaying expensive data operations can improve responsiveness if waiting allows smaller, highly selective data subsets to arrive first, reducing the total volume of data processed downstream.

EAGER MATERIALIZATION:
[Raw Data] ──> [Parse All Columns/Attributes] ──> [Memory Buffer] ──> [Apply Filter] ──> [Final Output]
                                                                        (Discarded Data Wasted Work)

LAZY MATERIALIZATION:
[Raw Data] ──> [Store Lightweight Offsets/IDs] ──> [Apply Filter] ──> [Parse Only Matching Rows] ──> [Final Output]

In modern GPU-accelerated relational query engines like Themis, Hong et al. demonstrate that lazy materialization yields substantial performance gains. Instead of eagerly loading all data attributes into limited GPU memory, Themis stores and manipulates lightweight data offsets, deferring the loading of actual attribute values until they are strictly necessary for the final output.

This philosophy applies across diverse data formats:

  • Hierarchical Views: Bohannon et al. utilize lazy materialization in ROLEX to generate portions of XML result trees on demand, guided by application navigational profiles, avoiding the overhead of pre-materializing massive structures.
  • Stream Processing: Winter et al. argue that eagerly parsing every column of an incoming data stream into internal buffers creates massive ingestion latency; transitioning to lazy parsing strategies ensures that only the specific attributes required by active queries are processed.
  • Stochastic Optimization: Quiané-Ruiz et al. apply this to gradient descent optimization by commuting transformation and sampling operators. By delaying data parsing and transformation until after a sampling step has identified the specific records needed for an update, the system avoids processing data points that are never utilized.
  • What-If Analysis: Balmin et al. demonstrate in SESAME that adopting a lazy evaluation strategy for hypothetical data states ensures computational resources are only expended on the specific data subsets required to answer a query, outperforming incremental update techniques for ad-hoc analysis.

Managing the State of Deferral

While deferred execution provides massive optimization windows, it introduces architectural complexity. Systems must carefully manage memory resources and internal state transitions while the graph is pending execution.

Salloum et al. highlight how Apache Spark’s engine constructs a DAG of transformations that are only triggered by an explicit action, allowing the system to analyze the entire chain for global optimization and proactive memory management. In distributed SPARQL query evaluation over RDF triples, Graux et al. show that lazy evaluation allows engines to manage intermediate results during complex multi-stage joins more effectively, preventing the memory failures that typically plague eager execution models.

Finally, because lazy abstractions decouple the invocation of code from its actual execution, they can introduce subtle state bugs. Läufer et al. emphasize that a state-pattern-based approach is necessary to manage internal transitions in lazy abstractions explicitly. By defining exactly how and when data is fetched, developers can ensure that elements are only processed when strictly required by the consumer, maintaining the integrity of the lazy contract.

Key Foundational Work

Hypothetical Queries in an OLAP Environment (2000) — contributed 1 concept elements to this direction.
Dynamic Query Optimization in Multidatabases (1997) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_lazy_materialization_ba03c756,
  title = {lazy materialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lazy-materialization/rdr_ba03c75643d84dfb9027173590f782aa},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_lazy_materialization_ba03c756_v1,
  title = {lazy materialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lazy-materialization/rdr_ba03c75643d84dfb9027173590f782aa/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake