Controlling False Discovery Rate Using Gaussian Mirrors

Xin Xing, Zhigen Zhao, Jun S. Liu

Abstract

Simultaneously finding multiple influential variables and controlling the false discovery rate (FDR) for linear regression models is a fundamental problem. We here propose the Gaussian Mirror (GM) method, which creates for each predictor variable a pair of mirror variables by adding and subtracting a randomly generated Gaussian perturbation, and proceeds with a certain regression method, such as the ordinary least-square or the Lasso (the mirror variables can also be created after selection). The mirror variables naturally lead to test statistics effective for controlling the FDR. Under a mild assumption on the dependence among the covariates, we show that the FDR can be controlled at any designated level asymptotically. We also demonstrate through extensive numerical studies that the GM method is more powerful than many existing methods for selecting relevant variables subject to FDR control, especially for cases when the covariates are highly correlated and the influential variables are not overly sparse.

Introduction

In modern data science, identifying influential variables within high-dimensional datasets—often referred to as the "needle in a haystack" problem—is a critical task. While maximizing discoveries is important, it is equally vital to control the False Discovery Rate (FDR), the expected proportion of false discoveries among all rejected null hypotheses.

A significant challenge arises when predictors (covariates) are highly correlated, a common occurrence in fields like genomics and finance. Traditional methods, such as the Knockoff Filter, attempt to address this by constructing "fake" variables that mimic the original data's correlation structure. However, in scenarios with dense and high correlations, generating valid knockoffs is computationally demanding and often leads to a substantial loss in statistical power.

The Gaussian Mirror Method

The Gaussian Mirror (GM) method introduces a novel approach to variable selection by leveraging symmetry. For each predictor variable $x_j$, the method constructs a pair of "mirror" variables by adding and subtracting a randomly generated Gaussian noise vector $z_j$:

  1. $x_j^+ = x_j + c_j z_j$
  2. $x_j^- = x_j - c_j z_j$

where $c_j$ is a scalar chosen to balance the perturbation.

Graphical representation of the Gaussian Mirror construction for variable $x_j$.

After constructing these mirrors, a regression model (e.g., OLS or Lasso) is fitted using the mirror variables. The importance of each original variable is then assessed using a mirror statistic, $M_j$, derived from the estimated coefficients $\hat{\beta}_j^+$ and $\hat{\beta}_j^-$: $$M_j = |\hat{\beta}_j^+ + \hat{\beta}_j^-| - |\hat{\beta}_j^+ - \hat{\beta}_j^-|$$

For a "null" variable (noise), the sum and difference of the mirror coefficients behave similarly, resulting in a distribution of $M_j$ that is symmetric around zero. Conversely, for a "relevant" variable (signal), the sum captures the signal while the difference cancels it out, leading to a large positive $M_j$. This symmetry allows for a data-driven thresholding process that guarantees FDR control by estimating false positives from the negative side of the $M_j$ distribution.

For high-dimensional scenarios where the number of predictors $p$ exceeds the sample size $n$, the paper introduces a post-selection adjustment strategy. This involves using Lasso for initial candidate screening followed by the GM method with a debiasing technique to maintain the required symmetry for FDR control.

Key Advantages

The GM method offers several advantages over existing frameworks:

  • Robustness to Correlation: Unlike the Knockoff Filter, which requires strict exchangeability, GM relies on a milder assumption regarding covariate dependence. This makes it significantly more effective in datasets with high correlations.
  • Computational Scalability: The construction of mirror statistics is inherently parallelizable, making the algorithm well-suited for modern high-performance computing environments.
  • Stability: The introduction of small perturbations leads to more stable variable selection, aligning with contemporary statistical theories on stability and reproducibility.

Empirical Results

The authors validated the GM method through extensive simulations and real-world data analysis.

Simulation Studies

In simulations featuring various correlation structures (e.g., power decay, constant positive correlation) and non-Gaussian distributions, the GM method consistently outperformed benchmarks. Notably, in high-correlation settings ($\kappa = 0.8$), GM maintained high discovery power while the Knockoff Filter's performance degraded significantly.

GWAS Tomato Dataset

The method was applied to a tomato Genome-Wide Association Study (GWAS) dataset involving over 11,000 SNPs. Despite the discrete nature of the predictors and complex genetic correlations, GM successfully identified relevant SNPs. Other methods were either too conservative or failed to control FDR effectively in this realistic setting.

Conclusion and Future Work

The Gaussian Mirror method provides a powerful and flexible framework for FDR-controlled variable selection. While the current theoretical guarantees are asymptotic, the empirical performance across diverse settings is promising. Future research may focus on optimizing the perturbation scale $c_j$ for non-linear models and extending the framework to generalized linear models (GLMs) and deep learning architectures (NGM).

Summary updated

Abstract

Simultaneously finding multiple influential variables and controlling the false discovery rate (FDR) for linear regression models is a fundamental problem. We here propose the Gaussian Mirror (GM) method, which creates for each predictor variable a pair of mirror variables by adding and subtracting a randomly generated Gaussian perturbation, and proceeds with a certain regression method, such as the ordinary least-square or the Lasso (the mirror variables can also be created after selection). The mirror variables naturally lead to test statistics effective for controlling the FDR. Under a mild assumption on the dependence among the covariates, we show that the FDR can be controlled at any designated level asymptotically. We also demonstrate through extensive numerical studies that the GM method is more powerful than many existing methods for selecting relevant variables subject to FDR control, especially for cases when the covariates are highly correlated and the influential variables are not overly sparse.
BibTeX
@article{xing2021controlling,
  author = {Xin Xing and Zhigen Zhao and Jun S. Liu},
  title = {{Controlling False Discovery Rate Using Gaussian Mirrors}},
  journal = {Journal of the American Statistical Association},
  year = {2021},
  doi = {10.1080/01621459.2021.1923510},
}
Figure