In traditional machine learning, a developer is responsible for both the "what" and the "how." They must define the model architecture (the what) and then manually implement the training loops, data shuffling, gradient descent, and distributed communication strategies (the how). This "imperative" approach forces researchers to spend more time on low-level "plumbing"—data munging and hardware-specific optimization—than on the actual learning objectives.
A growing body of research proposes a shift toward Declarative Machine Learning. In this paradigm, a user specifies the high-level learning goals, domain constraints, and data relationships, while a specialized system automatically determines the most efficient way to execute the task.
The Separation of Concerns
The core intuition behind declarative ML is borrowed from relational database systems. Just as a SQL user declares which data they want without specifying how to traverse B-trees or join tables, a declarative ML user defines the mathematical objective without writing the execution logic.
Christopher Jermaine argues that this separation of concerns allows the system to assume responsibility for optimizing mathematical operations and distributed execution. By decoupling the model specification from the underlying hardware, the same high-level code can scale from a single laptop to a massive cluster without manual rewriting. Ameet Talwalkar and the team behind MLbase frame this as treating machine learning like a database problem: the system uses a sophisticated optimizer to select algorithms and tune hyperparameters, abstracting away the implementation details.
Learning as a Logic Query
One powerful way to realize this abstraction is through formal logic. If a machine learning task can be expressed as a set of logical rules, the system can treat training as a query optimization problem.
Jin Wang and colleagues demonstrated this by using Datalog to implement machine learning algorithms. By treating training as a recursive logic query, their framework can automatically decide between different execution strategies, such as "broadcast" or "shuffle" joins. This approach often outperforms standard imperative frameworks like Spark’s MLlib because the compiler can optimize data flow and communication patterns specifically for the structure of the data, particularly in large-scale sparse datasets.
Similarly, the WOLFE stack, developed by Sebastian Riedel and Sameer Singh, allows users to define models using high-level mathematical operators like argmax and expectation. The system then maps these declarations to underlying inference algorithms, separating the model's mathematical definition from its execution.
Systems AI and Domain Knowledge
Declarative programming is not just about performance; it is about making complex AI systems easier to build and reason about. Parisa Kordjamshidi, Dan Roth, and Kristian Kersting advocate for a paradigm they call Declarative Learning-Based Programming (DeLBP).
In DeLBP, developers focus on "Systems AI"—integrating raw data with structured domain knowledge and constraints. Instead of hard-coding procedural logic, the developer declares the task structure and the rules the system must follow (e.g., "a person cannot be in two places at once"). The framework then automatically manages the "plumbing" of optimization and consistency checks. This allows for:
- Concise Code: Complex tasks, such as training hundreds of independent models across different data subsets, can be achieved with minimal overhead, as seen in the Saul language.
- Hybrid Reasoning: It bridges the gap between data-driven neural networks and symbolic reasoning, allowing models to learn from data while respecting human-defined logic.
End-to-End Optimization
The declarative approach can extend beyond the training loop to the entire data science lifecycle. Matthias Boehm and the SystemDS team developed a system that uses an optimizing compiler to manage everything from data cleaning to model debugging.
By using a unified declarative language, SystemDS eliminates the "boundary crossing" between different tools (like using SQL for cleaning and Python for training). The compiler can see the entire workflow and perform global optimizations, such as reusing intermediate results across different stages of the pipeline. This holistic view allows the system to speed up exploratory workflows that would otherwise be bottlenecked by manual data movement and redundant computations.