Group SLOPE – Adaptive Selection of Groups of Predictors

Damian Brzyski, Alexej Gossmann, Weijie Su, Małgorzata Bogdan

Abstract

Sorted L-One Penalized Estimation (SLOPE, Bogdan et al., 2013, 2015) is a relatively new convex optimization procedure which allows for adaptive selection of regressors under sparse high dimensional designs. Here we extend the idea of SLOPE to deal with the situation when one aims at selecting whole groups of explanatory variables instead of single regressors. Such groups can be formed by clustering strongly correlated predictors or groups of dummy variables corresponding to different levels of the same qualitative predictor. We formulate the respective convex optimization problem, gSLOPE (group SLOPE), and propose an efficient algorithm for its solution. We also define a notion of the group false discovery rate (gFDR) and provide a choice of the sequence of tuning parameters for gSLOPE so that gFDR is provably controlled at a prespecified level if the groups of variables are orthogonal to each other. Moreover, we prove that the resulting procedure adapts to unknown sparsity and is asymptotically minimax with respect to the estimation of the proportions of variance of the response variable explained by regressors from different groups. We also provide a method for the choice of the regularizing sequence when variables in different groups are not orthogonal but statistically independent and illustrate its good properties with computer simulations. Finally, we illustrate the advantages of gSLOPE in the context of Genome Wide Association Studies. R package grpSLOPE with an implementation of our method is available on CRAN.

1. Introduction

In modern high-dimensional data analysis—ranging from genomics to finance—predictors often exhibit natural group structures. For instance, multiple dummy variables may represent a single categorical factor, or genes may be grouped within biological pathways. While standard variable selection tools like the LASSO are effective for individual regressors, they struggle to maintain error control in group settings. Existing methods like the Group LASSO lack a reliable mechanism for controlling the False Discovery Rate (FDR) and are highly sensitive to tuning parameters that require prior knowledge of data sparsity. This paper introduces Group SLOPE (gSLOPE), a convex optimization framework that bridges group-based selection with rigorous error control and minimax optimality.

2. Methodology

2.1 The Sorted Penalty

The core innovation of gSLOPE is the extension of the Sorted L-One Penalized Estimation (SLOPE) to a group setting. Unlike standard Group LASSO, which applies a uniform penalty to all groups, gSLOPE utilizes a sequence of decreasing penalties $(\lambda_1, \lambda_2, \dots, \lambda_m)$. Groups are penalized based on the rank of their impact (norm); the group with the largest norm is penalized by $\lambda_1$, the second largest by $\lambda_2$, and so on. This sorting mechanism mimics the Benjamini-Hochberg procedure, allowing the model to dynamically adjust its discovery threshold and adapt to unknown sparsity levels.

2.2 Algorithmic Approach

To solve the resulting convex optimization problem, the authors employ an efficient Proximal Gradient Method (specifically a version of FISTA). The implementation also addresses practical hurdles, such as estimating the noise level ($\sigma$) when it is unknown, ensuring the method is robust for real-world applications.

3. Theoretical Guarantees and Empirical Results

The authors provide strong theoretical foundations for gSLOPE. Under orthogonal designs, the procedure provably controls the Group False Discovery Rate (gFDR) at a pre-specified level $q$. Furthermore, gSLOPE is shown to be asymptotically minimax for estimating group effects in sparse vectors, establishing a deep link between statistical testing and estimation risk.

Simulation results showing gFDR control (a, b) and statistical power (c) for different lambda sequences.

Extensive computer simulations validate these findings. As illustrated in the figure above, the "basic" $\lambda$ sequence derived from theory is often conservative, keeping the gFDR well below the target (Panel a). To enhance performance in non-orthogonal but independent designs, the authors introduced "corrected" or "relaxed" $\lambda$ sequences. These sequences maintain gFDR control (Panel b) while significantly increasing the power to detect true signals (Panel c), particularly as the number of relevant groups increases.

4. Discussion and Future Work

Group SLOPE represents a significant advancement in high-dimensional statistics by providing a sparsity-adaptive method with provable error control. While the current framework offers strong guarantees for orthogonal or independent groups, extending these results to designs with high inter-group correlation remains a key area for future research. Additionally, while the proximal gradient algorithm is efficient, the sorting step in each iteration adds a computational layer that may be further optimized for extremely large-scale datasets.

Researchers can implement these methods using the grpSLOPE R package available on CRAN.

Summary updated

Abstract

Sorted L-One Penalized Estimation (SLOPE, Bogdan et al., 2013, 2015) is a relatively new convex optimization procedure which allows for adaptive selection of regressors under sparse high dimensional designs. Here we extend the idea of SLOPE to deal with the situation when one aims at selecting whole groups of explanatory variables instead of single regressors. Such groups can be formed by clustering strongly correlated predictors or groups of dummy variables corresponding to different levels of the same qualitative predictor. We formulate the respective convex optimization problem, gSLOPE (group SLOPE), and propose an efficient algorithm for its solution. We also define a notion of the group false discovery rate (gFDR) and provide a choice of the sequence of tuning parameters for gSLOPE so that gFDR is provably controlled at a prespecified level if the groups of variables are orthogonal to each other. Moreover, we prove that the resulting procedure adapts to unknown sparsity and is asymptotically minimax with respect to the estimation of the proportions of variance of the response variable explained by regressors from different groups. We also provide a method for the choice of the regularizing sequence when variables in different groups are not orthogonal but statistically independent and illustrate its good properties with computer simulations. Finally, we illustrate the advantages of gSLOPE in the context of Genome Wide Association Studies. R package grpSLOPE with an implementation of our method is available on CRAN.
BibTeX
@article{brzyski2018group,
  author = {Damian Brzyski and Alexej Gossmann and Weijie Su and Małgorzata Bogdan},
  title = {{Group SLOPE – Adaptive Selection of Groups of Predictors}},
  journal = {Journal of the American Statistical Association},
  year = {2018},
  doi = {10.1080/01621459.2017.1411269},
}
Figure