An Elementary Proof of the Hanson-Wright Inequality

Ingvar ZiemannarXiv 2025 · DOI · arXiv

Abstract: The Hanson-Wright inequality establishes exponential concentration for quadratic forms $X^T M X$, where $X$ is a vector with independent sub-Gaussian entries and with parameters depending on the Frobenius and operator norms of $M$. The most elementary proof to date is due to Rudelson & Vershynin, who still rely on a convex decoupling argument due to Bourgain, followed by Gaussian comparison to arrive at the result. In this note we sidestep this decoupling and provide an arguably simpler proof reliant only on elementary properties of sub-Gaussian variables and Gaussian rotational invariance. As a consequence we also obtain improved constants.

Introduction

In high-dimensional statistics, random matrix theory, and compressed sensing, a fundamental challenge is understanding how quadratic forms of random vectors concentrate around their expected values. Specifically, for a random vector $X$ and a fixed matrix $M$, we seek to bound the probability that $X^T M X$ deviates significantly from $\mathbb{E}[X^T M X]$. The Hanson-Wright inequality provides this bound, showing that the deviation decays exponentially based on the matrix's Frobenius and operator norms.

While the inequality is a staple of the field, its derivation has traditionally relied on "convex decoupling"—a sophisticated technique used to handle dependencies between terms in the quadratic form. While effective, decoupling often acts as a "black box" that can obscure the underlying mechanics of the concentration. This paper presents a new, truly elementary proof that avoids decoupling entirely, relying instead on the basic definition of sub-Gaussianity and the rotational symmetry of Gaussian distributions.

Methodology

The proof strategy decomposes the problem into manageable components, avoiding complex machinery in favor of iterative comparison.

1. Matrix Decomposition

The matrix $M$ is first split into two parts: a diagonal matrix $D$ and a "hollow" matrix $A$ (where all diagonal entries are zero). The diagonal part $X^T D X$ is a sum of independent random variables, which can be bounded using standard Hoeffding-type inequalities. The primary challenge lies in the off-diagonal terms of the hollow matrix $A$.

2. Martingale Comparison and the Tower Rule

To handle the hollow matrix $A$, the author utilizes the "tower rule" of conditional expectation. By processing the vector $X$ entry-by-entry, the proof identifies a natural martingale structure. The core of the argument involves iteratively replacing each sub-Gaussian variable $X_i$ with a standard Gaussian variable $G_i$. Because sub-Gaussian variables are defined by their Moment Generating Function (MGF) being bounded by that of a Gaussian, this replacement provides a clean upper bound without the need for decoupling.

3. Gaussian Rotational Invariance

Once the problem is transformed into a Gaussian quadratic form, the author leverages the rotational invariance of Gaussian vectors. By "diagonalizing" the problem through a change of basis, the quadratic form is reduced to a weighted sum of independent chi-squared variables. This reduction allows for the application of the standard Hanson-Wright bound for Gaussian variables, which is well-understood.

4. Integration via Cauchy-Schwarz

Finally, the bounds for the diagonal and off-diagonal components are recombined using the Cauchy-Schwarz inequality, resulting in the final concentration bound for the original quadratic form.

Key Results

The resulting proof successfully derives the standard Hanson-Wright tail bound with improved numerical constants:

$$P(|X^T M X - \mathbb{E}[X^T M X]| \ge t) \le 2 \exp\left( - \min\left( \frac{t^2}{4c_1\sigma^4\|M\|_F^2}, \frac{t}{6c_2\sigma^2\|M\|_{op}} \right) \right)$$

Where:

  • $\|M\|_F$ is the Frobenius norm.
  • $\|M\|_{op}$ is the operator norm.
  • For diagonal-free (hollow) matrices, the constants are remarkably sharp: $c_1=2$ and $c_2=1$.
  • For general matrices, the constants are $c_1=20$ and $c_2=4$.

These constants are significantly tighter than those typically found in the literature, a direct result of avoiding the "lossy" steps inherent in decoupling arguments.

Discussion and Implications

By reducing the proof to two fundamental concepts—sub-Gaussian MGF properties and Gaussian rotational invariance—this approach offers significant pedagogical value. It demystifies a cornerstone of high-dimensional probability, making it more accessible to students and researchers who may not be specialists in decoupling techniques.

While the current proof assumes i.i.d. entries and focuses on sub-Gaussian distributions, its simplicity suggests potential for expansion. Future work may explore applying this iterative comparison method to more complex U-statistics or heavy-tailed distributions where decoupling was previously considered the only viable path forward.

Summary updated

Abstract

The Hanson-Wright inequality establishes exponential concentration for quadratic forms $X^T M X$, where $X$ is a vector with independent sub-Gaussian entries and with parameters depending on the Frobenius and operator norms of $M$. The most elementary proof to date is due to Rudelson & Vershinyn, who still rely on a convex decoupling argument due to Bourgain, followed by Gaussian comparison to arrive at the result. In this note we sidestep this decoupling and provide an arguably simpler proof reliant only on elementary properties of sub-Gaussian variables and Gaussian rotational invariance. As a consequence we also obtain improved constants.
BibTeX
@misc{ziemann2025elementary,
  author = {Ingvar Ziemann},
  title = {{An Elementary Proof of the Hanson-Wright Inequality}},
  year = {2025},
  eprint = {2509.00881},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.00881},
  howpublished = {arXiv},
}
Figure