Abstract
For the single index model $y=f(\beta^\tau x,\epsilon)$ with Gaussian design, satisfying that rank $var(\mathbb{E}[x\mid y])=1$ where $f$ is unknown and $\beta$ is a sparse $p$-dimensional unit vector with at most $s$ nonzero entries, we are interested in testing the null hypothesis that $\beta$, when viewed as a whole vector, is zero against the alternative that some entries of $\beta$ is nonzero. Assuming that $var(\mathbb{E}[x \mid y])$ is non-vanishing, we define the generalized signal-to-noise ratio (gSNR) $\lambda$ of the model as the unique non-zero eigenvalue of $var(\mathbb{E}[x \mid y])$. We show that if $s^{2}\log^2(p)\wedge p$ is of a smaller order of $n$, denoted as $s^{2}\log^2(p)\wedge p\prec n$, where $n$ is the sample size, one can detect the existence of signals if and only if gSNR$\succ\frac{p^{1/2}}{n}\wedge \frac{s\log(p)}{n}$. Furthermore, if the noise is additive (i.e., $y=f(\beta^\tau x)+\epsilon$), one can detect the existence of the signal if and only if gSNR$\succ\frac{p^{1/2}}{n}\wedge \frac{s\log(p)}{n} \wedge \frac{1}{\sqrt{n}}$. It is rather surprising that the detection boundary for the single index model with additive noise matches that for linear regression models. These results pave the road for thorough theoretical analysis of single/multiple index models in high dimensions.
Introduction
In modern high-dimensional statistics, the challenge of "global testing"—determining whether any relationship exists between a set of $p$ predictors and a response $y$—is fundamental. While the linear regression framework ($y = X\beta + \epsilon$) is well-understood, real-world data frequently exhibits non-linear dependencies. This research addresses the Single Index Model (SIM), a flexible semi-parametric framework defined as $y = f(\beta^\tau x, \epsilon)$.
In a SIM, the response depends on the predictors through a single linear combination $\beta^\tau x$, but the "link function" $f$ remains unknown. This flexibility allows the model to capture complex relationships (e.g., periodic, polynomial, or symmetric functions) that standard linear tests often miss. However, the lack of knowledge regarding $f$ introduces significant theoretical hurdles regarding identifiability and the fundamental limits of signal detection.
Defining the Signal: Generalized SNR
A primary contribution of this work is the formalization of the generalized Signal-to-Noise Ratio (gSNR), denoted as $\lambda$. In linear models, signal strength is typically tied to the magnitude of the coefficients. In the SIM context, where $f$ is unknown, the authors define $\lambda$ as the unique non-zero eigenvalue of $var(\mathbb{E}[x|y])$.
This metric captures how much the conditional distribution of the predictors $x$ shifts given the observed response $y$. By focusing on the variance of conditional expectations, the framework can detect signals even when the marginal correlation between $x$ and $y$ is zero—a common occurrence in symmetric non-linear functions like $y = (\beta^\tau x)^2$.
Methodology: Sliced Inverse Regression and Spectral Testing
To detect signals at the theoretical limit, the authors propose a framework based on Sliced Inverse Regression (SIR). SIR operates by partitioning the range of the response $y$ into "slices" and examining the sample mean of $x$ within each slice. This allows for the estimation of the structure of $var(\mathbb{E}[x|y])$ without requiring a specific form for $f$.
Two primary test statistics are introduced:
- SSS (Spectral test Statistic based on SIR): A general-purpose test for SIMs.
- SSSa (Augmented SSS): A specialized version for models with additive noise ($y = f(\beta^\tau x) + \epsilon$). This version incorporates an ANOVA-like component to enhance detection power.
To address the computational complexity of finding sparse eigenvalues in high dimensions (an NP-hard problem), the authors provide a computationally efficient implementation using Semi-Definite Programming (SDP) relaxation.
Theoretical Detection Boundaries
The paper establishes sharp "phase transitions" for signal detection. If the signal strength (gSNR) falls below a specific threshold, no statistical test can perform better than a random guess. Conversely, above this threshold, the proposed SSS tests succeed with high probability.
Assuming $s^{2}\log^2(p)\wedge p \prec n$, the detection boundary for the general SIM is: $$\text{gSNR} \succ \frac{p^{1/2}}{n} \wedge \frac{s \log p}{n}$$
Most strikingly, for models with additive noise, the boundary becomes: $$\text{gSNR} \succ \frac{p^{1/2}}{n} \wedge \frac{s \log p}{n} \wedge \frac{1}{\sqrt{n}}$$
This result reveals a "free lunch" in non-linear statistics: the fundamental difficulty of detecting a signal in a complex, non-linear additive model is identical to that of a simple linear regression. Not knowing the function $f$ does not impose a statistical penalty on the detection limit.
Empirical Validation
The theoretical findings were validated through extensive synthetic simulations using various link functions (exponential, sine, cubic) and covariance structures.
- Failure of Correlation: Standard tests like Higher Criticism (Cor-HC), which rely on marginal correlations, failed significantly when the link function was symmetric. In these cases, Cor-HC power remained near 0.10, while the SSS test maintained power levels between 0.90 and 1.00.
- Optimality: The SSS and SSa tests consistently achieved high power as soon as the gSNR crossed the predicted theoretical boundaries, confirming their rate-optimality.
Conclusion and Future Directions
This research provides a rigorous foundation for high-dimensional testing in non-linear settings. By proving that the detection limits for additive SIMs match those of linear models, the authors bridge a significant gap between parametric and semi-parametric theory.
Future work remains to extend these results beyond the Gaussian design assumption to more general sub-Gaussian distributions. Additionally, while this paper focuses on a single index, the framework provides a roadmap for analyzing Multiple Index Models, where the response depends on several linear combinations of the predictors.
Summary updated