An Empirical Comparison of Principal Component Analysis and Clustering on Variables for Dimension Reduction Using Leukemia and Breast Cancer Data

Khaled I. A. Almaghri, Saptarshi ChakrabortyInternational Journal of Statistics and Applications 2018

Abstract

Identifying nuisance variables that contribute to noise is a critical challenge in high-dimensional biological data analysis. This study compares two prominent dimension reduction techniques: Principal Component Analysis (PCA) and Variable Clustering. Using the Leukemia and German Breast Cancer Study Group 2 datasets, we evaluate each method's ability to retain information while simplifying data complexity. Our findings suggest that while PCA is mathematically efficient, Variable Clustering offers superior clinical interpretability by preserving original variables and enabling strategic selection based on data collection costs.

Introduction

In the era of "Big Data," biological datasets often contain hundreds of variables, many of which are "nuisance variables" that increase variability without adding predictive value. High dimensionality complicates analysis, slows down processing, and increases the financial and temporal costs of clinical data collection. Effective dimension reduction is therefore essential.

Principal Component Analysis (PCA), the traditional approach, creates synthetic, uncorrelated variables known as components. Conversely, Variable Clustering groups existing variables based on homogeneity, allowing researchers to select representative "real-world" variables from each cluster. This paper investigates which approach is more practical for clinicians and biostatisticians.

Methodology

The study compared two distinct philosophies of dimension reduction:

  1. Principal Component Analysis (PCA): A statistical technique that uses orthogonal transformation to turn a set of correlated variables into a smaller set of uncorrelated "Principal Components" (PCs). The goal is to capture the maximum variance in the fewest possible components.
  2. Variable Clustering: Instead of creating new variables, this method groups existing variables into clusters. The authors utilized the ClustOfVar package in R, employing bootstrap resampling (50 replications) to determine the stability and optimal number of clusters.

Results: Breast Cancer Analysis

The breast cancer dataset included observations on 686 women across variables such as tumor size, age, hormonal therapy, and progesterone receptor levels. PCA reduced 8 variables to 3 components, explaining 60.5% of the total variance. However, this reduction resulted in a significant (nearly 40%) loss of information and failed to include "tumor grade" in the primary reduced set.

Scree plot for breast cancer data showing eigenvalues for principal components.

In contrast, Variable Clustering identified 5 distinct clusters. This method allowed for a more nuanced selection of representatives—such as age, tumor size, and progesterone receptor status—without the same level of abstract information loss. The hierarchical grouping of these clinical variables is visualized in the dendrogram below.

Dendrogram showing the hierarchical clustering of breast cancer variables.

Results: Leukemia (ALL) Analysis

The Leukemia (ALL) dataset, consisting of microarray data from 128 individuals across 11 variables, showed similar trends. PCA reduced the data to 5 components, explaining 77.4% of the variance (a 22.6% information loss). Variable Clustering identified 8 stable clusters, providing a more granular and interpretable view of the underlying molecular biology markers.

Discussion: The Interpretability Gap

The primary advantage of Variable Clustering over PCA in a medical context is interpretability. PCA's synthetic components are mathematical abstractions that do not correspond directly to clinical tests; a physician cannot easily interpret a "Principal Component" in a diagnostic setting.

Variable Clustering, however, identifies groups of real variables that behave similarly. This enables Strategic Variable Selection: if two variables provide the same information, a researcher can choose the one that is cheaper, faster, or less invasive to obtain (e.g., patient age vs. a complex lab-based molecular marker). By keeping the most representative real-world variables from each cluster, researchers maintain a high level of data integrity that PCA’s synthetic components cannot match.

Conclusion

Variable clustering is recommended for medical research where data integrity and interpretability are paramount. It provides a pathway for dimension reduction that maintains a direct link to clinical measurements. While PCA remains a powerful tool for general variance explanation, the practical benefits of clustering—specifically the ability to select cost-effective, interpretable variables—make it a superior choice for biostatistical applications.

Future research should involve feeding these reduced datasets into predictive models, such as survival analysis or diagnostic classifiers, to quantitatively prove that clustered variables outperform principal components in live clinical environments.

Summary updated

Abstract

One of the important problems of data analysis is that identifying nuisance variable(s) in a data set that contributes to an increase of variability within groups in an experiment. One way to address this issue is through dimension reduction of data sets. In this study we compare between two widely used methods of reducing dimension data sets, namely the method of the principal component (PC), statistics technical that uses orthogonal transformation to convert a set of possibly correlated variables of into a new set of uncorrelated variables and the method of clustering on variables, where the aim is to put the variables with similar information in the same group or cluster by considering two celebrated data sets from literature, the leukemia dataset and the other a breast cancer data.
BibTeX
@article{almaghri2018empirical,
  author = {Khaled I. A. Almaghri and Saptarshi Chakraborty},
  title = {{An Empirical Comparison of Principal Component Analysis and Clustering on Variables for Dimension Reduction Using Leukemia and Breast Cancer Data}},
  journal = {International Journal of Statistics and Applications},
  year = {2018},
}
Figure