Information Theoretic Meta Learning with Gaussian Processes
Lists
Only you see this
Abstract
We formulate meta learning using information theoretic concepts such as mutual information and the information bottleneck. The idea is to learn a stochastic representation or encoding of the task description, given by a training or support set, that is highly informative about predicting the validation set. By making use of variational approximations to the mutual information, we derive a general and tractable framework for meta learning. We particularly develop new memory-based meta learning algorithms based on Gaussian processes and derive extensions that combine memory and gradient-based meta learning. We demonstrate our method on few-shot regression and classification by using standard benchmarks such as Omniglot, mini-Imagenet and Augmented Omniglot.
BibTeX
@inproceedings{titsias2021information,
author = {Michalis Titsias and Sotirios Nikoloutsopoulos and Alexandre Galashov},
title = {{Information Theoretic Meta Learning with Gaussian Processes}},
booktitle = {Conference on Uncertainty in Artificial Intelligence},
year = {2021},
}