Unified Optimization of Relational Queries and Machine Learning Inference

This direction is merged into another direction: in-database machine learning. Its last version stays readable and citable here.

Version 1 of 19 papers16 conceptsfrozen 2026-10-04

Traditionally, data science workflows are split into two silos: the database (RDBMS), which filters and joins raw data, and the machine learning (ML) framework, which performs inference. This "wall" between data management and model execution creates a massive bottleneck. Data must be exported from the database, transformed into a format the ML model understands, and then processed—often ignoring the fact that the database already knows a great deal about the data's structure and constraints.

A new research direction seeks to tear down this wall by treating ML inference as a first-class relational operator. By integrating models directly into the query engine, systems can apply classic database optimization techniques to the models themselves, leading to orders-of-magnitude improvements in speed and cost.

ML Models as Relational Operators

The core intuition behind this shift is that an ML model—whether it is a decision tree, a linear regression, or even a Large Language Model (LLM)—can be viewed as a functional transformation within a data pipeline. In systems like Raven, developed by Konstantinos Karanasos, Matteo Interlandi, and their colleagues, this is achieved through a unified Intermediate Representation (IR).

When the database engine sees both the SQL query and the ML model in the same IR, it no longer treats the model as an opaque "black box." Instead, it can optimize the entire pipeline holistically. This allows the system to perform "cross-optimizations" that were previously impossible when the two worlds were separate.

Cross-Layer Optimization: Predicate-Aware Pruning

One of the most powerful results of this integration is predicate-aware model pruning. Imagine a SQL query that asks for predictions only for customers older than 50:

SELECT predict_churn(model) FROM customers WHERE age > 50;

In a traditional setup, the database filters the rows and sends them to the model. The model then evaluates every internal branch for every row. However, as shown in Extending Relational Query Processing with ML Inference, if the model is a decision tree that contains a split on age < 30, the unified engine can statically determine that the "under 30" branch will never be reached for this specific query.

By "pruning" these unreachable branches before execution, the system reduces the computational complexity of the model based on the query's filters. Karanasos et al. demonstrated that this can speed up inference by up to 24x, simply by ensuring the model doesn't waste cycles on logic that the relational predicates have already ruled out.

Declarative AI and Cost-Based Planning

The integration of AI into relational engines extends beyond classical ML to modern generative models. In A Declarative System for Optimizing AI Workloads, Chunwei Liu, Mike Cafarella, and their team introduce PALIMPZEST, a system that treats LLM calls as relational operators.

The challenge with LLMs is that they are expensive and slow. However, by using a declarative approach, the system can automatically optimize how these "operators" are executed. For example:

  1. Model Selection: The optimizer can choose between a high-quality, expensive model (like GPT-4) and a faster, cheaper one based on the user's budget and accuracy requirements.
  2. Predicate Pushdown: If a user wants to extract information from documents but only cares about documents from a specific year, the system can "push" that filter down, ensuring the expensive LLM only processes the relevant subset of data.

By abstracting away the manual labor of prompt engineering and model selection, PALIMPZEST can achieve up to a 90x speedup over naive implementations. The system balances cost, runtime, and data quality by generating an execution plan that treats AI tasks with the same rigor as a standard SQL join.

Why It Works: The Power of Context

The success of this research direction stems from a simple realization: inference does not happen in a vacuum. Every time a model is run, it is run on a specific subset of data for a specific purpose.

By unifying the relational engine and the inference engine, we allow the "context" of the data (the filters, the types, the distributions) to inform the "execution" of the model. Whether it is pruning a decision tree branch in Raven or selecting a cheaper LLM in PALIMPZEST, the goal is the same: use relational knowledge to eliminate redundant AI computation.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_unified_optimization_of_relational_queri_e02a9ae0,
  title = {Unified Optimization of Relational Queries and Machine Learning Inference},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unified-optimization-of-relational-queries-and-machine-learning-inference/rdr_e02a9ae06c3047458f161cacd28ef185},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_unified_optimization_of_relational_queri_e02a9ae0_v1,
  title = {Unified Optimization of Relational Queries and Machine Learning Inference},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unified-optimization-of-relational-queries-and-machine-learning-inference/rdr_e02a9ae06c3047458f161cacd28ef185/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}