On strict sub-Gaussianity, optimal proxy variance and symmetry for\n bounded random variables

Julyan Arbel, Olivier Marchal, Hien D. Nguyen

Introduction

In probability theory and machine learning, sub-Gaussian distributions are highly valued for their "thin" tails, which allow for the application of powerful concentration inequalities. While it is well-established that any random variable $X$ with almost surely bounded support is sub-Gaussian, determining the most precise parameters for these bounds remains a significant challenge.

This research addresses two fundamental questions regarding bounded random variables:

  1. What is the optimal proxy variance ($\sigma^2_{opt}$)? This is the smallest possible value that can be used in sub-Gaussian inequalities to provide the tightest possible bounds.
  2. When is a distribution "strictly" sub-Gaussian? A distribution is strictly sub-Gaussian if its optimal proxy variance is exactly equal to its standard variance ($\sigma^2_{opt} = \text{Var}[X]$).

The $h(\lambda)$ Framework

To characterize these properties, the authors move beyond traditional tail-bound proofs and focus on the cumulant generating function (CGF). They introduce a variational approach centered on a function $h(\lambda)$, derived from the CGF, where the global maximum of $h(\lambda)$ over all real numbers $\lambda$ corresponds to the optimal proxy variance $\sigma^2_{opt}$.

To analyze this function, the researchers employed several technical strategies:

  • Variational Analysis: Studying the derivatives and variations of $h(\lambda)$ to identify its extrema.
  • Ordinary Differential Equations (ODEs): The authors derived specific ODEs that $h$ must satisfy, allowing them to prove the uniqueness of global maxima for specific distributions.
  • Taylor Expansions: By comparing the Taylor series of a distribution’s transform against a Gaussian transform, they established necessary conditions for strict sub-Gaussianity based on kurtosis.

Debunking the Symmetry Myth

One of the most significant contributions of this paper is the refutation of a common intuition: that symmetry is the primary driver of strict sub-Gaussianity. In Proposition 1.2, the authors demonstrate that symmetry is neither necessary nor sufficient for a distribution to be strictly sub-Gaussian.

They provide counter-examples, including a symmetric mixture of Dirac masses that is not strictly sub-Gaussian and an asymmetric mixture that is. Instead of symmetry, the authors point to kurtosis as a critical factor. For a distribution to be strictly sub-Gaussian, it must be platykurtic (having an excess kurtosis $\le 0$). If a distribution has "fat" tails relative to a Gaussian, it cannot be strictly sub-Gaussian, regardless of its boundedness or symmetry.

Analysis of Standard Distributions

The paper applies this framework to several fundamental probability distributions, providing new proofs and formulas:

  • Bernoulli and Binomial: The authors resolved an open problem regarding the uniqueness of the global maximum for the Bernoulli proxy variance. They proved that a Bernoulli distribution is strictly sub-Gaussian if and only if it is symmetric ($p = 1/2$). For asymmetric cases, they provided a closed-form formula for the optimal proxy variance.
  • Uniform Distribution: The study confirms that the uniform distribution is strictly sub-Gaussian, with $\sigma^2_{opt} = \text{Var}[X] = 1/12$.
  • Triangular Distribution: These distributions are strictly sub-Gaussian if and only if they are symmetric ($a=b$).
  • Kumaraswamy Distribution: This class is found to be strictly sub-Gaussian only in the specific case where it simplifies to the uniform distribution.

Conclusion and Future Directions

By transforming the search for proxy variance into a function maximization problem, the authors have created a rigorous "recipe" for analyzing bounded variables. While this work focuses on univariate bounded support, it opens several avenues for future research. Extending these variational techniques to unbounded sub-Gaussian variables and characterizing optimal proxy covariance matrices for multivariate distributions (such as the Dirichlet distribution) remain promising areas for exploration. Currently, some complex distributions still require numerical methods to find $\sigma^2_{opt}$, suggesting that further refinement of closed-form solutions is needed.

Summary updated

Abstract

We investigate the sub-Gaussian property for almost surely bounded random\nvariables. If sub-Gaussianity per se is de facto ensured by the bounded support\nof said random variables, then exciting research avenues remain open. Among\nthese questions is how to characterize the optimal sub-Gaussian proxy variance?\nAnother question is how to characterize strict sub-Gaussianity, defined by a\nproxy variance equal to the (standard) variance? We address the questions in\nproposing conditions based on the study of functions variations. A particular\nfocus is given to the relationship between strict sub-Gaussianity and symmetry\nof the distribution. In particular, we demonstrate that symmetry is neither\nsufficient nor necessary for strict sub-Gaussianity. In contrast, simple\nnecessary conditions on the one hand, and simple sufficient conditions on the\nother hand, for strict sub-Gaussianity are provided. These results are\nillustrated via various applications to a number of bounded random variables,\nincluding Bernoulli, beta, binomial, uniform, Kumaraswamy, and triangular\ndistributions.\n
BibTeX
@inproceedings{arbel2019strict,
  author = {Julyan Arbel and Olivier Marchal and Hien D. Nguyen},
  title = {{On strict sub-Gaussianity, optimal proxy variance and symmetry for\textbackslash\{\}n bounded random variables}},
  booktitle = {ESAIM},
  year = {2019},
  doi = {10.1051/ps/2019018},
}
Figure