Feature Screening via Distance Correlation Learning

Runze Li, Wei Zhong, Li Zhu

TL;DR

This paper introduces a new feature screening method called DC-SIS, which uses distance correlation to identify important features in ultra-high dimensional data. DC-SIS is more powerful than existing methods, especially when relationships are non-linear, and can handle complex scenarios like grouped predictors and multiple response variables without needing to specify a model.

What Problem Does This Paper Address?

In modern data analysis, we often encounter datasets with a massive number of features (predictors) compared to the number of observations. This "ultra-high dimensionality" poses a significant challenge for traditional statistical and machine learning methods, which can struggle with computational efficiency, statistical accuracy, and stability. Feature screening aims to identify a smaller subset of relevant features that are most likely to be related to the outcome of interest, making subsequent analysis more manageable and effective. Existing methods like Sure Independence Screening (SIS) based on Pearson correlation work well for linear relationships but can miss important features if the relationship is non-linear. They also often require specifying a model for the data, which can be difficult in ultra-high dimensional settings where the underlying structure is unknown.

Methods

The core innovation of this paper is the development of the Distance Correlation Sure Independence Screening (DC-SIS) procedure. Distance correlation is a statistical measure of dependence between two random variables that can capture non-linear relationships, unlike Pearson correlation. The DC-SIS works by calculating the distance correlation between each predictor variable and the response variable. Predictors with a higher distance correlation are considered more important.

Key aspects of the method include:

  • Model-Free: DC-SIS does not require a pre-specified model (e.g., linear or generalized linear) for the relationship between predictors and the response. This is a significant advantage in ultra-high dimensional data where model specification is challenging.
  • Generalizability: The "sure screening property" (ensuring that truly important predictors are selected with high probability) is proven for DC-SIS under more general settings than SIS, including non-linear relationships.
  • Handling Complex Data: DC-SIS can be directly applied to screen grouped predictor variables (e.g., dummy variables representing categories in ANOVA) and to handle multivariate response variables, which are limitations of many existing methods.

Datasets

The paper uses simulated datasets for its numerical studies. It also illustrates the method with a real-world dataset: the Cardiomyopathy microarray dataset.

Key Insights and Ideas

  • Distance Correlation is a Powerful Tool for Feature Screening: The ability of distance correlation to capture non-linear dependencies makes DC-SIS a more robust feature screening method than those relying solely on linear correlation.
  • Model-Free Screening is Crucial for Ultra-High Dimensionality: The lack of a model specification requirement makes DC-SIS highly practical and less prone to errors due to incorrect model assumptions.
  • Unified Approach for Complex Data Structures: DC-SIS offers a single framework to handle grouped predictors and multivariate responses, simplifying analysis in these scenarios.
  • Improved Performance in Non-Linear Settings: Simulations show DC-SIS significantly outperforms traditional methods when the underlying relationships are non-linear.

Key Results

  • Theoretical Guarantees: The paper establishes the "sure screening property" for DC-SIS, meaning it can identify relevant features with high probability as the sample size grows.
  • Superior Performance in Simulations: Monte Carlo simulations demonstrate that DC-SIS significantly outperforms SIS and SIRS (another screening method) in various models, especially those with non-linear relationships, interactions, or heteroscedasticity.
  • Effective for Grouped and Multivariate Data: The method shows strong performance in selecting grouped predictors and in scenarios with multivariate responses, as illustrated in the examples.
  • Real-World Application: The DC-SIS successfully identified important genes in the Cardiomyopathy microarray dataset, showing better performance than a generalized correlation ranking method in explaining the response variable.

Limitations and Future Work

  • Choice of Cutoff: The paper acknowledges that the choice of the cutoff threshold (d) for screening is important and can affect performance, similar to tuning parameters in regularization methods. This is identified as an area for future research.
  • Iterative Procedure: Like SIS, DC-SIS might miss predictors that are marginally independent of the response. The authors mention an earlier version of their paper proposed an iterative DC-SIS to address this, suggesting further theoretical study is needed for such iterative approaches.
  • Marginally Independent Predictors: Developing new methods to identify predictors that are marginally independent of the response but still important in a multivariate context is highlighted as an important future research direction.

Summary updated

Abstract

This article is concerned with screening features in ultrahigh-dimensional data analysis, which has become increasingly important in diverse scientific fields. We develop a sure independence screening procedure based on the distance correlation (DC-SIS). The DC-SIS can be implemented as easily as the sure independence screening (SIS) procedure based on the Pearson correlation proposed by Fan and Lv. However, the DC-SIS can significantly improve the SIS. Fan and Lv established the sure screening property for the SIS based on linear models, but the sure screening property is valid for the DC-SIS under more general settings, including linear models. Furthermore, the implementation of the DC-SIS does not require model specification (e.g., linear model or generalized linear model) for responses or predictors. This is a very appealing property in ultrahigh-dimensional data analysis. Moreover, the DC-SIS can be used directly to screen grouped predictor variables and multivariate response variables. We establish the sure screening property for the DC-SIS, and conduct simulations to examine its finite sample performance. A numerical comparison indicates that the DC-SIS performs much better than the SIS in various models. We also illustrate the DC-SIS through a real-data example.
BibTeX
@article{li2012feature,
  author = {Runze Li and Wei Zhong and Li Zhu},
  title = {{Feature Screening via Distance Correlation Learning}},
  journal = {Journal of the American Statistical Association},
  year = {2012},
  doi = {10.1080/01621459.2012.695654},
}

Loading related content...

Figure