A Theory of Universal Agnostic Learning

Steve Hanneke, Shay Moran

Abstract

We provide a complete theory of optimal universal rates for binary classification in the agnostic setting. This extends the realizable-case theory of Bousquet, Hanneke, Moran, van Handel, and Yehudayoff (2021) by removing the realizability assumption on the distribution. We identify a fundamental tetrachotomy of optimal rates: for every concept class, the optimal universal rate of convergence of the excess error rate is one of $e^{-n}$, $e^{-o(n)}$, $o(n^{-1/2})$, or arbitrarily slow. We further identify simple combinatorial structures which determine which of these categories any given concept class falls into.
BibTeX
@misc{hanneke2026theory,
  author = {Steve Hanneke and Shay Moran},
  title = {{A Theory of Universal Agnostic Learning}},
  year = {2026},
  eprint = {2601.20961},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.20961},
}

Loading related content...

Figure