Abstract
In high-dimensional statistics, quantifying sparsity is crucial for effective estimation. Traditional approaches rely on $\ell^p$-norms (for $0 \le p < 2$), yet these measures often fail to provide sharp lower bounds for minimax risk. This paper introduces the truncated $\ell^2$-norm as a robust alternative that "caps" the contribution of large signal components at the noise level. We demonstrate that this norm provides a sharper characterization of estimation difficulty, enables the development of adaptive thresholding estimators, and offers new insights into the consistency of Bayesian priors in high dimensions.
Introduction
The assumption of sparsity—that a signal consists primarily of zero or near-zero components—is the cornerstone of modern high-dimensional data analysis. While the $\ell^0$-norm (counting non-zeros) and the $\ell^1$-norm (the basis for the Lasso) are widely used, theoretical gaps remain. Specifically, while upper bounds for risk are well-established, lower bounds—representing the fundamental limits of performance—are difficult to generalize across the full range of sparsity levels. This research proposes the truncated $\ell^2$-norm as a more natural and mathematically "honest" metric for signals embedded in Gaussian noise.
The Geometry of Sparsity
The core innovation is the truncated $\ell^2$-norm, defined for an $n$-dimensional vector $\theta$ as: $$\|\theta\|_{t,\sigma^2} = \left( \sum_{i=1}^n \min(\theta_i^2, \sigma^2) \right)^{1/2}$$ This norm effectively ignores the magnitude of signal components once they exceed the noise level $\sigma$, focusing instead on the cumulative impact of small and moderate components.
Geometrically, the truncated $\ell^2$-ball encompasses traditional $\ell^p$-balls. As shown in Figure 1, the $\ell^1$-ball (yellow diamond) and the $\ell^0$-ball (red axes) are subsets of the truncated $\ell^2$-ball (blue region). This inclusion property is significant: it implies that any theoretical performance guarantee established for the truncated norm automatically applies as a "safety net" for more restrictive $\ell^p$ measures. Furthermore, the paper proves that estimating a signal within a truncated $\ell^2$-ball is asymptotically as difficult as estimating one in an $\ell^0$-ball, positioning the truncated norm as an ideal continuous proxy for discrete sparsity.
Minimax Risk and Adaptive Estimation
Using the $n$-dimensional Gaussian location model ($x = \theta + \epsilon$), the author derives exact asymptotic limits for minimax risk. For a signal with a truncated norm $\eta$, the minimax risk scales precisely as $2\eta \log(\eta^{-1})$. This is a "sharp" result, meaning the constants are exact.
Building on this theoretical foundation, the paper introduces an adaptive thresholding estimator. Unlike traditional methods that require prior knowledge of the sparsity level, this estimator uses a data-driven surrogate ($\kappa_t$) to estimate the truncated norm and dynamically adjusts the threshold $\lambda$. This approach is shown to be asymptotically minimax across all $\ell^p$-balls ($0 \le p < 2$) simultaneously, provided the signal is sufficiently detectable above the noise floor.
Bayesian Consistency in High Dimensions
The framework also extends to Bayesian statistics, providing a new lower bound for Bayes risk. This bound serves as a diagnostic tool for evaluating high-dimensional priors. A key finding is that for a Bayesian prior to be consistent (where error vanishes as data increases), its "variance-standardized truncated norm" must vanish.
Applying this to popular models, the research reveals that the Laplace prior (used in the Bayesian Lasso) can become inconsistent in high dimensions under certain conditions. In contrast, the Horseshoe prior demonstrates superior robustness, with its risk scaling more gracefully relative to its hyperparameters.
Conclusion and Future Work
The truncated $\ell^2$-norm offers a unified perspective on sparsity, bridging the gap between discrete counts and continuous shrinkage. While currently focused on Gaussian location models, the potential for extending this framework to high-dimensional regression ($y = X\beta + \epsilon$) and non-Gaussian noise distributions represents a promising frontier for future research. Current limitations include the requirement for finite variance in Bayesian lower bounds and the specific focus on the simple location model.
Summary updated