Introduction
In probability theory and machine learning, sub-Gaussian distributions are highly valued for their "thin" tails, which allow for the application of powerful concentration inequalities. While it is well-established that any random variable $X$ with almost surely bounded support is sub-Gaussian, determining the most precise parameters for these bounds remains a significant challenge.
This research addresses two fundamental questions regarding bounded random variables:
- What is the optimal proxy variance ($\sigma^2_{opt}$)? This is the smallest possible value that can be used in sub-Gaussian inequalities to provide the tightest possible bounds.
- When is a distribution "strictly" sub-Gaussian? A distribution is strictly sub-Gaussian if its optimal proxy variance is exactly equal to its standard variance ($\sigma^2_{opt} = \text{Var}[X]$).
The $h(\lambda)$ Framework
To characterize these properties, the authors move beyond traditional tail-bound proofs and focus on the cumulant generating function (CGF). They introduce a variational approach centered on a function $h(\lambda)$, derived from the CGF, where the global maximum of $h(\lambda)$ over all real numbers $\lambda$ corresponds to the optimal proxy variance $\sigma^2_{opt}$.
To analyze this function, the researchers employed several technical strategies:
- Variational Analysis: Studying the derivatives and variations of $h(\lambda)$ to identify its extrema.
- Ordinary Differential Equations (ODEs): The authors derived specific ODEs that $h$ must satisfy, allowing them to prove the uniqueness of global maxima for specific distributions.
- Taylor Expansions: By comparing the Taylor series of a distribution’s transform against a Gaussian transform, they established necessary conditions for strict sub-Gaussianity based on kurtosis.
Debunking the Symmetry Myth
One of the most significant contributions of this paper is the refutation of a common intuition: that symmetry is the primary driver of strict sub-Gaussianity. In Proposition 1.2, the authors demonstrate that symmetry is neither necessary nor sufficient for a distribution to be strictly sub-Gaussian.
They provide counter-examples, including a symmetric mixture of Dirac masses that is not strictly sub-Gaussian and an asymmetric mixture that is. Instead of symmetry, the authors point to kurtosis as a critical factor. For a distribution to be strictly sub-Gaussian, it must be platykurtic (having an excess kurtosis $\le 0$). If a distribution has "fat" tails relative to a Gaussian, it cannot be strictly sub-Gaussian, regardless of its boundedness or symmetry.
Analysis of Standard Distributions
The paper applies this framework to several fundamental probability distributions, providing new proofs and formulas:
- Bernoulli and Binomial: The authors resolved an open problem regarding the uniqueness of the global maximum for the Bernoulli proxy variance. They proved that a Bernoulli distribution is strictly sub-Gaussian if and only if it is symmetric ($p = 1/2$). For asymmetric cases, they provided a closed-form formula for the optimal proxy variance.
- Uniform Distribution: The study confirms that the uniform distribution is strictly sub-Gaussian, with $\sigma^2_{opt} = \text{Var}[X] = 1/12$.
- Triangular Distribution: These distributions are strictly sub-Gaussian if and only if they are symmetric ($a=b$).
- Kumaraswamy Distribution: This class is found to be strictly sub-Gaussian only in the specific case where it simplifies to the uniform distribution.
Conclusion and Future Directions
By transforming the search for proxy variance into a function maximization problem, the authors have created a rigorous "recipe" for analyzing bounded variables. While this work focuses on univariate bounded support, it opens several avenues for future research. Extending these variational techniques to unbounded sub-Gaussian variables and characterizing optimal proxy covariance matrices for multivariate distributions (such as the Dirichlet distribution) remain promising areas for exploration. Currently, some complex distributions still require numerical methods to find $\sigma^2_{opt}$, suggesting that further refinement of closed-form solutions is needed.
Summary updated