Abstract
We consider estimating the shared mean of a sequence of heavy-tailed random variables taking values in a Banach space. In particular, we revisit and extend a simple truncation-based mean estimator first proposed by Catoni and Giulini. While existing truncation-based approaches require a bound on the raw (non-central) second moment of observations, our results hold under a bound on either the central or non-central $p$th moment for some $p \in (1,2]$. Our analysis thus handles distributions with infinite variance. The main contributions of the paper follow from exploiting connections between truncation-based mean estimation and the concentration of martingales in smooth Banach spaces. We prove two types of time-uniform bounds on the distance between the estimator and unknown mean: line-crossing inequalities, which can be optimized for a fixed sample size $n$, and iterated logarithm inequalities, which match the tightness of line-crossing inequalities at all points in time up to a doubly logarithmic factor in $n$. Our results do not depend on the dimension of the Banach space, hold under martingale dependence, and all constants in the inequalities are known and small.
Introduction
In classical statistics, the sample mean is the standard tool for central tendency estimation. However, its reliability relies heavily on two assumptions: that the data has light tails (finite variance) and that observations are independent and identically distributed (i.i.d.). In modern applications such as reinforcement learning, high-frequency finance, and multi-armed bandits, these assumptions are frequently violated. Data often exhibits "heavy tails" where outliers are common and variance may be infinite, and observations are often sequentially dependent.
Whitehouse et al. (2025) address these challenges by developing a robust, computationally efficient mean estimator designed for complex mathematical environments (Banach spaces). Their approach extends truncation-based estimation to handle infinite variance and martingale dependence while providing "anytime-valid" confidence bounds.
The Truncation-Based Estimator
The core of the proposed method is a truncation mechanism. Unlike a standard average, which is highly sensitive to extreme values, a truncation-based estimator "squishes" data points that fall beyond a specific threshold. While truncation is a known technique, this paper introduces several critical innovations that improve its utility in high-dimensional and dependent settings.
Two-Stage Centering and Translation Invariance
A significant limitation of traditional truncation methods is their lack of translation invariance; if the data is shifted, the estimator’s performance changes. This occurs because previous methods typically truncated data around the origin.
The authors solve this by employing a two-stage centering process. They use a small initial batch of data to generate a "naive" preliminary estimate. The subsequent observations are then truncated relative to this preliminary estimate rather than zero. This shift ensures the estimator is translation-invariant and allows the error bounds to depend on central moments (variance-like measures) rather than raw moments, significantly tightening the resulting confidence intervals.
Dimension-Free Estimation in Banach Spaces
The analysis extends beyond Euclidean space to 2-smooth Banach spaces, which include Hilbert spaces and $L^\alpha$ spaces. A key feature of this work is that the results are dimension-free. The error bounds depend on the geometric "smoothness" of the space rather than its dimensionality, making the estimator particularly effective for high-dimensional data where traditional methods often suffer from the "curse of dimensionality."
Handling Dependence and Anytime-Validity
Most robust estimators assume independence, but real-world sequential data is rarely i.i.d. The authors utilize a martingale dependence structure, where the conditional mean of the next observation is the true mean given the history.
To make the estimator practical for sequential decision-making, the authors apply a "stitching" technique. This transforms fixed-sample-size bounds into time-uniform (anytime-valid) bounds. These bounds remain valid at every time step $n$, allowing researchers to stop an experiment early or monitor results in real-time without compromising the statistical integrity of the confidence intervals.
Empirical Performance
The authors validated their theoretical findings through controlled simulations using Pareto II (Lomax) distributions. By setting the shape parameter to $a = 1.75$, they created a scenario with infinite variance—a regime where the standard sample mean fails to converge reliably.
Key findings from the simulations include:
- Optimal Convergence: The estimator achieves a convergence rate of $O(n^{-(p-1)/p})$. When $p=2$ (finite variance), this matches the standard $1/\sqrt{n}$ rate, but it remains robust even when $p < 2$.
- Superior Tightness: When initialized with a geometric median-of-means, the truncation-based estimator provided tighter confidence bounds than both the sample mean and Minsker’s geometric median-of-means estimator.
- Computational Efficiency: Unlike "tournament" or "median-of-means" estimators that require solving complex optimization problems, this estimator is essentially a weighted average. It is computationally light and suitable for online updates.
Conclusion and Future Work
This work provides a rigorous framework for mean estimation in the presence of heavy tails and dependence. By explicitly calculating the constants in their inequalities, the authors ensure the bounds are not just theoretically interesting but practically applicable.
Future research directions include making the estimator adaptive so it can learn the optimal truncation level without prior knowledge of the $p$-th moment. Additionally, while the current analysis is optimized for the very heavy-tailed regime ($1 < p \leq 2$), extending the framework to capture faster sub-Gaussian rates for $p > 2$ remains an open challenge.
Summary updated