Declarative Machine Learning Systems

47 concepts33 papers1 proposal

In traditional machine learning, building a model forces the developer to act as a low-level systems plumber. They must write imperative code that explicitly dictates how a model should execute: initializing weight matrices, manually writing training loops, managing data sharding across distributed GPU clusters, and hardcoding hyperparameter tuning routines. This tight coupling of mathematical logic and physical execution creates immense technical debt, makes code brittle across changing hardware environments, and limits machine learning to specialized systems engineers.

Declarative machine learning flips this paradigm. Borrowing a foundational concept from relational database management systems (RDBMS), it separates the logical specification of a machine learning task from its physical execution. Instead of writing procedural training code, the user provides a high-level declaration of intent—defining what the data looks like and what task needs to be solved—and leaves it to an underlying compiler and query optimizer to determine the most efficient way to train, optimize, and deploy the model.

As noted by Christopher Jermaine et al. in Letter from the Special Issue Editor, this approach mirrors the classic separation of concerns found in SQL databases. By delegating the complexities of mathematical execution and distributed optimization to the system, declarative ML reduces the engineering burden on practitioners and allows workflows to scale seamlessly across diverse hardware environments.


The Core Mechanism: Logic-Execution Separation

To understand why declarative machine learning works, we must look at how these systems ingest and transform user intent. In a declarative system, the user's input is not an executable script but a high-level schema or mathematical declaration.

For example, in the Overton system introduced by Christopher Re in Overton: A Data System for Monitoring and Improving Machine-Learned Products, engineers do not write deep learning code or manipulate model architectures. Instead, they define a high-level schema specifying the data payloads and the tasks they want to accomplish. The system automatically compiles this schema into a multitask deep learning architecture.

[ User Schema / Intent ]  -->  [ Declarative Compiler ]  -->  [ Optimized Physical Execution Plan ]
(Data shapes & tasks)          (Architecture selection)        (Distributed GPUs / SQL Backends)

Because the logical task definition is completely decoupled from the physical model implementation, system developers can upgrade the underlying neural architectures or optimize hardware backends without requiring users to modify a single line of their application logic (Declarative machine learning systems).

This abstraction can manifest in several ways depending on the system's design:


The Query Optimizer as a Distributed Systems Engineer

The true power of declarative ML lies in what happens after the user specifies their intent. Because the system possesses a high-level, abstract representation of the computation, it can perform global optimizations that are incredibly tedious to implement manually.

Consider the challenge of distributing a massive machine learning workload across a cluster of machines. In an imperative framework, an engineer must manually write code to shard data, broadcast weights, or implement model-parallel partitioning. If the cluster configuration changes or the dataset grows, the code often breaks or suffers from severe performance degradation.

Declarative systems solve this by treating machine learning like a database query optimization problem. In MLbase, designed by Ameet Talwalkar et al. in MLbase: A Distributed Machine-learning System, the system maps high-level learning tasks to an optimized execution strategy, automatically selecting algorithms and tuning parameters while leveraging relaxed consistency models to accelerate training.

[ Declarative Query ] 
        │
        ▼
[ Query Optimizer ] ───► Reads Data Statistics & Cluster Topography
        │
        ▼
[ Dynamic Execution Plan ] ───► Chooses Data-Parallel vs. Model-Parallel Strategies

This database-centric optimization reaches its peak when machine learning is embedded directly into relational computations. Yuxin Tang et al. demonstrated in Auto-Differentiation of Relational Computations for Very Large Scale Machine Learning that by redefining relational algebra operations as differentiable functions, automatic differentiation can be performed directly inside databases.

In this setup, the standard database query optimizer effectively serves as an automated distributed systems engineer. It reads data statistics and dynamically chooses between data-parallel and model-parallel strategies (such as broadcasting or co-partitioning) to distribute gradient computations across a cluster. Similarly, SimSQL, developed by Dimitrije Jankov et al. in Declarative Recursive Computation on an RDBMS, uses declarative recursive queries to automatically determine optimal data distribution, allowing massive models to train successfully on datasets that cause traditional imperative frameworks to crash.


Engine Independence and Compilation

Because a declarative specification describes what to compute rather than how to compute it, it remains entirely independent of any specific execution engine. This allows a single model specification to be compiled into completely different target environments without manual code rewrites.

The ML2SQL framework, presented by Maximilian E. Schüle et al. in ML2SQL - Compiling a Declarative Machine Learning Language to SQL and Python, exemplifies this capability. They introduce a declarative language called MLearn that allows models to be defined using high-level constructs like tensors and control blocks. The ML2SQL compiler can take this exact same model specification and translate it into either Python (utilizing NumPy and TensorFlow) or directly into SQL stored procedures (for PostgreSQL or HyPer). This allows high-performance machine learning to happen exactly where the data lives, eliminating the need to export massive datasets over the network to specialized ML environments.

                      ┌───► Python (NumPy / TensorFlow)
                      │
[ MLearn Specification ] ───► [ ML2SQL Compiler ] ──┤
                      │
                      └───► SQL Stored Procedures (In-Database)

This engine-agnostic nature also enables the integration of machine learning directly into standard data analysis workflows. As argued by Beng Chin Ooi et al. in Towards a Unified Graph Model for Supporting Data Management and Usable Machine Learning, extending declarative query languages with native machine learning operators allows non-experts to perform complex analytical tasks like classification or clustering within their queries, completely separating the logical intent of the workflow from the physical execution details.


Taming Production Complexity

In production environments, writing the core machine learning algorithm is often a minor part of the overall engineering challenge. The majority of real-world complexity stems from managing the lifecycle of the model: tracking metadata, ensuring reproducibility, managing technical debt, and optimizing resource utilization across shared infrastructure.

Imperative pipelines obscure these operational details inside opaque, custom code blocks. Declarative pipelines, by contrast, expose the structure of the workflow directly to the underlying system. As Sebastian Schelter et al. explain in On Challenges in Machine Learning Model Management, defining pipelines by their intended outcomes rather than their imperative execution steps allows systems to automatically extract metadata, track data lineage, and apply cross-hardware optimizations. This structural transparency drastically reduces technical debt and ensures long-term reproducibility.

Furthermore, in multi-tenant environments like shared enterprise GPU clusters, declarative specifications allow the system to optimize resource allocation globally. When multiple users submit workloads to a platform like Ease.ml (Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads), the system does not just blindly run their code. Because it understands the data shapes and tasks via declarative schemas, it can employ advanced scheduling algorithms—combining Bayesian optimization with multi-armed bandits—to manage cluster resources and maximize global model quality up to an order of magnitude faster than traditional human-led heuristics.

Key Foundational Work

MLbase: A Distributed Machine-learning System (2013) — contributed 2 concept elements to this direction.
A Gobal View of Automatic Programming (1973) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_declarative_machine_learning_bff275df,
  title = {declarative machine learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/declarative-machine-learning/rdr_bff275df959147ff81d8510df4f02728},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_declarative_machine_learning_bff275df_v1,
  title = {declarative machine learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/declarative-machine-learning/rdr_bff275df959147ff81d8510df4f02728/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake