The Balanced Accuracy and Its Posterior Distribution

Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, Joachim M. Buhmann

Abstract

Evaluating the performance of a classification algorithm critically requires a measure of the degree to which unseen examples have been identified with their correct class labels. In practice, generalizability is frequently estimated by averaging the accuracies obtained on individual cross-validation folds. This procedure, however, is problematic in two ways. First, it does not allow for the derivation of meaningful confidence intervals. Second, it leads to an optimistic estimate when a biased classifier is tested on an imbalanced dataset. We show that both problems can be overcome by replacing the conventional point estimate of accuracy by an estimate of the posterior distribution of the balanced accuracy.
BibTeX
@inproceedings{brodersen2010balanced,
  author = {Kay Henning Brodersen and Cheng Soon Ong and Klaas Enno Stephan and Joachim M. Buhmann},
  title = {{The Balanced Accuracy and Its Posterior Distribution}},
  booktitle = {ICPR},
  year = {2010},
  doi = {10.1109/icpr.2010.764},
}

Loading related content...

Figure