Optimal sub-Gaussian variance proxy for truncated Gaussian and exponential random variables

M.t. Barreto, Olivier Marchal, Julyan ArbelStatistics 2025 · DOI · arXiv

TL;DR

This paper provides the first closed-form solutions for the "optimal variance proxy" of truncated Gaussian and exponential random variables. The authors prove that while truncated Gaussians are "strictly" sub-Gaussian only when truncated symmetrically, truncated exponentials never achieve this property, offering a new theoretical foundation for more precise concentration inequalities in machine learning and statistics.

What Problem Does This Paper Address?

In fields like reinforcement learning, Bayesian statistics, and financial modeling, we often deal with "truncated" data—data that is forced to live within a specific range (e.g., a sensor that caps at a maximum value or a clinical trial with a specific age range).

Mathematically, these variables are often sub-Gaussian, meaning their "tails" decay at least as fast as a normal distribution. This property is the engine behind concentration inequalities (like Hoeffding’s inequality), which tell us how much a sample mean is likely to deviate from the true mean.

However, there is a catch: to use these inequalities effectively, you need a variance proxy ($s^2$). While any large number will technically work as a proxy, using one that is too large leads to "loose" bounds and overly conservative estimates. The "holy grail" is the optimal variance proxy—the smallest possible value that satisfies the sub-Gaussian definition. Until now, exact formulas for these proxies were missing for two of the most common distributions in science: the truncated Gaussian and the truncated exponential.

Methods

The paper is purely theoretical, relying on mathematical proofs rather than simulations. The authors characterize the optimal variance proxy as the unique solution to a system of two equations derived from the Laplace condition (the log-moment generating function).

Key steps in their approach include:

  1. Reduction to Standard Forms: They show that finding the proxy for any truncated Gaussian or exponential can be simplified to finding it for the "standard" versions (mean 0, variance 1 for Gaussian; mean 1 for exponential).
  2. Geometric Tangency: They frame the search for the optimal proxy as a geometric problem: finding a parabola that is perfectly tangent to the log-moment generating function of the distribution.
  3. Analytical Proofs: They use calculus and properties of convexity/concavity to prove that their proposed closed-form solutions are indeed the unique, optimal values.

Datasets

Not discussed in the paper. As a theoretical probability paper, it focuses on deriving universal formulas rather than testing on specific empirical datasets.

Key Insights and Ideas

  • The Symmetry Rule for Gaussians: The authors discover a beautiful "aha moment" regarding strict sub-Gaussianity (where the optimal proxy equals the actual variance). For truncated Gaussians, this happens if and only if the truncation is symmetric around the mean. If you cut the distribution lopsidedly, the variance proxy must be strictly larger than the variance to account for the resulting skewness.
  • Exponentials are Never "Strict": Unlike the Gaussian case, a truncated exponential distribution is never strictly sub-Gaussian. No matter how you truncate it, the optimal variance proxy will always be larger than the variance.
  • Closed-Form Accessibility: Perhaps the most practical insight is that these proxies aren't just abstract concepts; the authors provide explicit, plug-and-play formulas (Theorems 2.1 and 3.1) that researchers can now use to sharpen their own bounds.
  • The "Divergence" of Exponentials: They show that as the upper truncation bound of an exponential variable goes to infinity, the variance proxy also goes to infinity. This confirms why the standard (untruncated) exponential distribution isn't sub-Gaussian at all.

Key Results

  • Theorem 2.1 (Gaussian): Provides the exact formula for $\|X\|_{vp}^2$ based on the truncation points $a$ and $b$, the mean $\mu$, and the variance $\sigma^2$. It confirms that if $a+b = 2\mu$, the proxy is simply the variance.
  • Theorem 3.1 (Exponential): Provides the exact formula for the exponential proxy. For a standard exponential truncated on $[\alpha, \beta]$, the proxy is $\frac{(\beta - \alpha)(e^\alpha + e^\beta)}{2(e^\beta - e^\alpha)} - 1$.
  • Visual Proofs: The paper includes elegant visualizations (Figures 1 and 3) showing the "gap" between variance and the variance proxy, illustrating exactly where the "strict" property holds or fails.

Limitations and Future Work

While this paper closes the book on two major distributions, the authors acknowledge several paths forward:

  • Beyond Gaussian/Exponential: The same "tangency" method could be applied to other common distributions like the Gamma or Weibull families.
  • Multivariate Truncation: Real-world data is often multi-dimensional. Extending these optimal proxies to truncated multivariate Gaussians remains a significant challenge.
  • Sub-Weibull Properties: The authors suggest exploring "sub-Weibull" properties, which are a generalization of sub-Gaussianity for distributions with even heavier tails.

In short, this work provides the "missing constants" that will allow statisticians and machine learning engineers to move from "safe but loose" bounds to "optimal and tight" ones when dealing with constrained data.

Summary updated

BibTeX
@inproceedings{barreto2025optimal,
  author = {M.T. Barreto and Olivier Marchal and Julyan Arbel},
  title = {{Optimal sub-Gaussian variance proxy for truncated Gaussian and exponential random variables}},
  booktitle = {Statistics},
  year = {2025},
  doi = {10.1016/j.spl.2025.110555},
}
Figure