MLJ: A Julia package for composable machine learning

Anthony D. Blaom, Franz J. Király, Thibaut Liénart, Yiannis Simillides, Diego Arenas, Sebastian J. Vollmer

TL;DR

MLJ (Machine Learning in Julia) is a comprehensive, open-source machine learning framework that provides a unified interface for training, tuning, and evaluating models. Its standout feature is a highly flexible "learning networks" API that enables advanced model composition—such as branching pipelines, learned target transformations, and model stacking—overcoming the rigid linear pipeline limitations found in traditional tools like scikit-learn.

What Problem Does This Paper Address?

Modern data science relies heavily on high-level ML toolboxes like scikit-learn (Python) or mlr/caret (R) to blueprint and deploy models. However, as practitioners build increasingly sophisticated composite models (e.g., the complex stacked ensembles seen in Kaggle competitions), these traditional frameworks reveal limitations. They typically restrict users to linear (non-branching) pipelines, struggle with out-of-sample predictions for base learners, and make it difficult to couple hyperparameters across homogeneous ensembles.

Furthermore, these frameworks suffer from the "two-language problem": they are prototyped in high-level languages (Python/R) but rely on low-level languages (C/Fortran) for performance, making it difficult for users to inspect, modify, or apply automatic differentiation to the underlying algorithms. MLJ addresses these architectural and linguistic bottlenecks by providing a cohesive, highly composable ML ecosystem written entirely in Julia.

Methods

MLJ introduces several novel software design patterns for machine learning:

  • Learning Networks (Advanced Composability): Instead of simple linear pipelines, MLJ uses Directed Acyclic Graphs (DAGs) to define composite models. This allows for branching architectures, model stacking, and dynamic target transformations.
  • The Model vs. Machine Abstraction: In MLJ, a model is just a struct containing hyperparameters. To train, a model is bound to data inside a machine, which stores the learned parameters. This separation allows multiple nodes in a network to share the same learned parameters or hyperparameters.
  • Smart Retraining (Warm Restarts): Because composite models are defined as DAGs, MLJ performs lazy evaluation. If a user changes a hyperparameter at the end of a pipeline (e.g., tweaking a classifier), MLJ is smart enough to only retrain that specific component, skipping the expensive retraining of upstream data transformers.
  • Scientific Types: MLJ abstracts machine data types (e.g., Float32) into semantic "scientific types" (e.g., Continuous, Multiclass). This ensures models are only fed data they are mathematically designed to handle.

Datasets

Not discussed in the paper. As this is a software architecture and design overview, the authors focus on API design, syntax, and framework capabilities rather than benchmarking against specific datasets.

Key Insights and Ideas

  • First-Class Probabilistic Predictions: Instead of returning raw arrays of probabilities (which forces users to guess conventions like class ordering or standard deviation vs. variance), MLJ's probabilistic models return actual sampleable Distribution objects. This unifies how metrics like cross-entropy or Brier score are calculated.
  • Safe Categorical Tracking: A common pain point in ML is evaluating a test set that contains a categorical class not seen in the training set, causing integer-recasting frameworks to crash. MLJ uses CategoricalArrays.jl to embed the entire pool of possible classes into the data representation, preserving labels and preventing these crashes.
  • Solving the Two-Language Problem: Because MLJ and its underlying models are written in Julia, users get the ease of a high-level language with the speed of C. This enables powerful features out-of-the-box, such as applying Automatic Differentiation (AD) directly to the model code.

Key Results

Because this paper introduces a software package, it does not contain quantitative benchmark results (like accuracy or speedup metrics). The primary "results" are qualitative architectural achievements:

  • Successfully implemented an API where composite models inherit all the behaviors of ordinary models.
  • Created a unified hyperparameter tuning interface (TunedModel) that acts as a simple model wrapper, allowing tuning to be seamlessly injected anywhere in a complex workflow.
  • Established a model registry that allows users to query model metadata and data requirements without having to load the actual model code into memory.

Limitations and Future Work

While the authors do not explicitly detail the framework's current limitations, it is implied that the ecosystem of available models is still growing compared to mature giants like scikit-learn.

For future work, the MLJ team plans to:

  • Expand the "model zoo" (the number of supported algorithms).
  • Add an update_data method to the model interface to natively support online/streaming learning.
  • Broaden the framework's scope to include probabilistic programming models, time series analysis, sparse data support, and natural language processing (NLP).

Summary updated

Abstract

MLJ (Machine Learing in Julia) is an open source software package providing a common interface for interacting with machine learning models written in Julia and other languages. It provides tools and meta-algorithms for selecting, tuning, evaluating, composing and comparing those models, with a focus on flexible model composition. In this design overview we detail chief novelties of the framework, together with the clear benefits of Julia over the dominant multi-language alternatives.
BibTeX
@article{blaom2020mlj,
  author = {Anthony D. Blaom and Franz J. Király and Thibaut Liénart and Yiannis Simillides and Diego Arenas and Sebastian J. Vollmer},
  title = {{MLJ: A Julia package for composable machine learning}},
  journal = {J. Open Source Softw.},
  year = {2020},
  url = {https://openreview.net/forum?id=JeGiWZRsH_n},
}

Loading related content...

Figure