Transfer Learning for Bayesian HPO with End-to-End Meta-Features
Lists
Only you see this
Abstract
Hyperparameter optimization (HPO) is a crucial component of deploying machine learning models, however, it remains an open problem due to the resource-constrained number of possible hyperparameter evaluations. As a result, prior work focus on exploring the direction of transfer learning for tackling the sample inefficiency of HPO. In contrast to existing approaches, we propose a novel Deep Kernel Gaussian Process surrogate with Landmark Meta-features (DKLM) that can be jointly meta-trained on a set of source tasks and then transferred efficiently on a new (unseen) target task. We design DKLM to capture the similarity between hyperparameter configurations with an end-to-end meta-feature network that embeds the set of evaluated configurations and their respective performance. As a result, our novel DKLM can learn contextualized dataset-specific similarity representations for hyperparameter configurations. We experimentally validate the performance of DKLM in a wide range of HPO meta-datasets from OpenML and demonstrate the empirical superiority of our method against a series of state-of-the-art baselines.
BibTeX
@inproceedings{jomaa2022transfer,
author = {Hadi Samer Jomaa and Sebastian Pineda Arango and Lars Schmidt-Thieme and Josif Grabocka},
title = {{Transfer Learning for Bayesian HPO with End-to-End Meta-Features}},
booktitle = {International Conference on Learning Representations},
year = {2022},
url = {https://openreview.net/forum?id=wronZ3Mx_d},
}Loading related content...