LTF: A Label Transformation Framework for Correcting Label Shift

Jiaxian Guo, Mingming Gong, Tongliang Liu, Kun Zhang, Dacheng Tao

Abstract

Distribution shift is a major obstacle to the deployment of current deep learning models on real-world problems. Let π‘Œ be the class label and 𝑋 the features. We focus on one type of distribution shift, \emph{ label shift}, where the label marginal distribution π‘ƒπ‘Œ changes but the conditional distribution 𝑃𝑋|π‘Œ does not. Most existing methods estimate the density ratio between the source- and target-domain label distributions by density matching. However, these methods are either computationally infeasible for large-scale data or restricted to shift correction for discrete labels. In this paper, we propose an end-to-end Label Transformation Framework (LTF) for correcting label shift, which implicitly models the shift of π‘ƒπ‘Œ and the conditional distribution 𝑃𝑋|π‘Œ using neural networks. Thanks to the flexibility of deep networks, our framework can handle continuous, discrete, and even multi-dimensional labels in a unified way and is scalable to large data. Moreover, for high dimensional 𝑋, such as images, we find that the redundant information in 𝑋 severely degrades the estimation accuracy. To remedy this issue, we propose to match the distribution implied by our generative model and the target-domain distribution in a low-dimensional feature space that discards information irrelevant to π‘Œ. Both theoretical and empirical studies demonstrate the superiority of our method over previous approaches.
BibTeX
@inproceedings{guo2020ltf,
  author = {Jiaxian Guo and Mingming Gong and Tongliang Liu and Kun Zhang and Dacheng Tao},
  title = {{LTF: A Label Transformation Framework for Correcting Label Shift}},
  booktitle = {International Conference on Machine Learning},
  year = {2020},
}
Figure