DeepPipe: Deep, Modular and Extendable Representations of Machine Learning Pipelines

Sebastian Pineda Arango, Josif GrabockaICLR 2023 · OpenReview

Abstract

Finding accurate Machine Learning pipelines is essential in achieving state-of-the-art AI predictive performance. Unfortunately, most existing Pipeline Optimization techniques rely on flavors of Bayesian Optimization that do not explore the deep interaction between pipeline stages/components (e.g. between hyperparameters of the deployed preprocessing algorithm and the hyperparameters of a classifier). In this paper, we are the first to capture the deep interaction between components of a Machine Learning pipeline. We propose embedding pipelines in a deep latent representation through a novel per-component encoder mechanism. Such pipeline embeddings are used with deep kernel Gaussian Process surrogates inside a Bayesian Optimization setup. Through extensive experiments on large-scale meta-datasets, we demonstrate that learning pipeline embeddings with Deep Neural Networks significantly advances the state-of-the-art in Pipeline Optimization.
BibTeX
@inproceedings{arango2023deeppipe,
  author = {Sebastian Pineda Arango and Josif Grabocka},
  title = {{DeepPipe: Deep, Modular and Extendable Representations of Machine Learning Pipelines}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=TT66Tpbus3b},
}

Loading related content...

Figure