On divergences, surrogate loss functions and decentralized detection

XuanLong Nguyen, Martin J. Wainwright, Michael I. Jordan

Abstract

We develop a general correspondence between a family of loss functions that act as surrogates to 0-1 loss, and the class of Ali-Silvey or $f$-divergence functionals. This correspondence provides the basis for choosing and evaluating various surrogate losses frequently used in statistical learning (e.g., hinge loss, exponential loss, logistic loss); conversely, it provides a decision-theoretic framework for the choice of divergences in signal processing and quantization theory. We exploit this correspondence to characterize the statistical behavior of a nonparametric decentralized hypothesis testing algorithms that operate by minimizing convex surrogate loss functions. In particular, we specify the family of loss functions that are equivalent to 0-1 loss in the sense of producing the same quantization rules and discriminant functions.
BibTeX
@misc{nguyen2005divergences,
  author = {XuanLong Nguyen and Martin J. Wainwright and Michael I. Jordan},
  title = {{On divergences, surrogate loss functions and decentralized detection}},
  year = {2005},
  eprint = {math/0510521},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/math/0510521},
}

Loading related content...

Figure