On the Importance of Feature Separability in Predicting Out-Of-Distribution Error
Lists
Only you see this
Abstract
Estimating the generalization performance is practically challenging on out-of-distribution (OOD) data without ground-truth labels. While previous methods emphasize the connection between distribution difference and OOD accuracy, we show that a large domain gap not necessarily leads to a low test accuracy. In this paper, we investigate this problem from the perspective of feature separability empirically and theoretically. Specifically, we propose a dataset-level score based upon feature dispersion to estimate the test accuracy under distribution shift. Our method is inspired by desirable properties of features in representation learning: high inter-class dispersion and high intra-class compactness. Our analysis shows that inter-class dispersion is strongly correlated with the model accuracy, while intra-class compactness does not reflect the generalization performance on OOD data. Extensive experiments demonstrate the superiority of our method in both prediction performance and computational efficiency.
BibTeX
@inproceedings{xie2023importance,
author = {Renchunzi Xie and Hongxin Wei and Lei Feng and Yuzhou Cao and Bo An},
title = {{On the Importance of Feature Separability in Predicting Out-Of-Distribution Error}},
booktitle = {Advances in Neural Information Processing Systems},
year = {2023},
url = {https://openreview.net/forum?id=A86JTXllHa},
}