MLJ: A Julia package for composable Machine Learning

Anthony D. Blaom, Franz J. Király, Thibaut Liénart, Yiannis Simillides, Diego Arenas, Sebastian J. Vollmer

Abstract

MLJ (Machine Learning in Julia) is an open source software package providing a common interface for interacting with machine learning models written in Julia and other languages. It provides tools and meta-algorithms for selecting, tuning, evaluating, composing and comparing those models, with a focus on flexible model composition. In this design overview, we detail chief novelties of the framework, together with the clear benefits of Julia over the dominant multi-language alternatives.

Introduction

Modern data science relies heavily on high-level machine learning toolboxes like scikit-learn (Python) or mlr/caret (R) to blueprint and deploy models. However, as practitioners build increasingly sophisticated composite models—such as complex stacked ensembles—these traditional frameworks reveal architectural limitations. They typically restrict users to linear, non-branching pipelines and struggle with out-of-sample predictions for base learners. Furthermore, these frameworks suffer from the "two-language problem": they are prototyped in high-level languages but rely on low-level languages for performance, making it difficult for users to inspect or modify underlying algorithms. MLJ addresses these bottlenecks by providing a cohesive, highly composable ML ecosystem written entirely in Julia.

Methods and Design Patterns

MLJ introduces several novel software design patterns to enhance flexibility and performance in machine learning workflows.

Learning Networks and Advanced Composability

Instead of simple linear pipelines, MLJ uses Directed Acyclic Graphs (DAGs) to define composite models. This "learning networks" API allows for branching architectures, model stacking, and dynamic target transformations. Because composite models are defined as DAGs, MLJ performs lazy evaluation. If a user changes a hyperparameter at the end of a pipeline, MLJ is smart enough to only retrain that specific component, skipping expensive upstream data transformations.

The Model vs. Machine Abstraction

In MLJ, a model is a struct containing only hyperparameters. To perform training, a model is bound to data inside a machine, which stores the learned parameters. This separation allows multiple nodes in a network to share the same learned parameters or hyperparameters, facilitating complex ensemble methods.

Training and prediction workflows illustrating the separation of models and machines.

Scientific Types

MLJ abstracts machine data types (e.g., Float32) into semantic "scientific types" (e.g., Continuous, Multiclass). This ensures models are only fed data they are mathematically designed to handle, providing a layer of safety and clarity in data preprocessing.

Hierarchy of scientific types in MLJ.

Key Insights

  • First-Class Probabilistic Predictions: MLJ's probabilistic models return actual sampleable Distribution objects rather than raw arrays of probabilities. This unifies the calculation of metrics like cross-entropy or Brier scores across different model types.
  • Safe Categorical Tracking: A common pain point in ML is evaluating test sets with categories not seen during training. MLJ uses CategoricalArrays.jl to embed the entire pool of possible classes into the data representation, preserving labels and preventing runtime crashes.
  • Solving the Two-Language Problem: Because MLJ and its underlying models are written in Julia, users benefit from the ease of a high-level language combined with the execution speed of C. This enables advanced features such as applying Automatic Differentiation (AD) directly to model code.

Conclusion and Future Work

MLJ establishes a unified API where composite models inherit all behaviors of ordinary models, supported by a model registry for efficient metadata querying. While the ecosystem is still growing, the primary achievement is a framework that overcomes the rigid limitations of traditional ML tools. Future work will focus on expanding the "model zoo," supporting online/streaming learning via an update_data method, and broadening the scope to include probabilistic programming and time series analysis.

Summary updated

Abstract

MLJ (Machine Learing in Julia) is an open source software package providing a common interface for interacting with machine learning models written in Julia and other languages. It provides tools and meta-algorithms for selecting, tuning, evaluating, composing and comparing those models, with a focus on flexible model composition. In this design overview we detail chief novelties of the framework, together with the clear benefits of Julia over the dominant multi-language alternatives.
BibTeX
@article{blaom2020mlj,
  author = {Anthony D. Blaom and Franz J. Király and Thibaut Liénart and Yiannis Simillides and Diego Arenas and Sebastian J. Vollmer},
  title = {{MLJ: A Julia package for composable Machine Learning}},
  journal = {J. Open Source Softw.},
  year = {2020},
  url = {https://openreview.net/forum?id=JeGiWZRsH_n},
}
Figure