Multiple output samples for each input in a single-output Gaussian process

Jeremy Wong, Huayun Zhang, Nancy F. ChenICLR 2023 · OpenReview

Abstract

The standard Gaussian Process (GP) is formulated to only consider a single output sample for each input in the training set. Datasets for subjective tasks, such as spoken language assessment, may be annotated with output labels from multiple human raters for each input. This paper proposes to generalise the GP to allow for multiple output samples per input in the training set. This differs from a multi-output GP, because all output samples are from the same task here. The output density function is formulated to be the joint likelihood of observing all output samples. Through this, the hyper-parameters are optimised using a criterion that is similar to minimising a Kullback-Leibler divergence. The test set predictions are inferred fairly similarly to a standard GP, with a key difference being in the optimised hyper-parameters. This approach is evaluated on spoken language assessment tasks, using the public speechocean762 dataset and an internal Tamil language dataset. The results show that by using the proposed method, the GP is able to compute a test set output distribution that is more similar to the collection of reference outputs annotated by multiple human raters.
BibTeX
@inproceedings{wong2023multiple,
  author = {Jeremy Wong and Huayun Zhang and Nancy F. Chen},
  title = {{Multiple output samples for each input in a single-output Gaussian process}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=jNpvW1ozbj3},
}

Loading related content...

Figure