Supervised Discriminative Sparse PCA for Com-Characteristic Gene Selection and Tumor Classification on Multiview Biological Data

Chun-Mei Feng, Yong Xu, Jin-Xing Liu, Ying-Lian Gao, Chun-Hou Zheng

TL;DR

Researchers have developed Supervised Discriminative Sparse PCA (SDSPCA), a new dimensionality reduction method that combines the interpretability of sparse PCA with the predictive power of supervised learning. By forcing the model to "see" class labels while selecting a tiny subset of relevant genes, SDSPCA significantly outperforms existing methods in classifying tumors and identifying "com-characteristic" genes across multiple types of cancer.

What Problem Does This Paper Address?

In the world of genomics, researchers often deal with the "curse of dimensionality": datasets might contain expression levels for over 20,000 genes, but only a few hundred samples. Standard Principal Component Analysis (PCA) is the go-to tool for simplifying this data, but it has two major flaws for clinicians:

  1. Lack of Supervision: PCA is unsupervised; it finds directions of maximum variance, which don't always correspond to the differences between a healthy cell and a tumorous one.
  2. Poor Interpretability: Standard PCA produces "dense" results, meaning every single gene contributes a tiny bit to the final model. Biologists need "sparse" results—a short list of specific genes they can actually study in a lab.

Furthermore, biological data is often "multiview," meaning it comes from different sources or disease types. The authors aim to find com-characteristic genes—markers that are consistently important across different cancer types—to better understand shared disease pathways.

Methods

The authors introduce SDSPCA, which modifies the core PCA objective function by adding two critical constraints:

  • Discriminative Supervision: They incorporate a class indicator matrix (labels) directly into the optimization. This forces the principal components to not just represent the data's variance, but to actively approximate the labels of the samples.
  • $L_{2,1}$-Norm Sparsity: Unlike traditional sparse PCA, which usually puts sparsity constraints on the "loadings" (the weights), SDSPCA imposes sparsity on the components themselves. This acts as a feature selection mechanism, effectively "turning off" irrelevant genes.

The team developed an iterative algorithm to solve this complex optimization problem and provided a mathematical proof that the algorithm consistently converges to a stable solution.

Datasets

The model was evaluated using high-dimensional gene expression data from The Cancer Genome Atlas (TCGA).

  • Multiview Data: The study combined data from three specific cancers: Pancreatic cancer (PAAD), Head and Neck squamous cell carcinoma (HNSC), and Cholangiocarcinoma (CHOL).
  • Scale: The dataset included 20,502 genes across 643 samples, categorized into four classes (the three cancers plus normal tissue).

Key Insights and Ideas

  • Sparse Components as "Codes": By imposing sparsity on components rather than loadings, the authors treat the reduced data as "sparse codes." This makes the physical interpretation of the results much clearer for biological gene selection.
  • Reducing Class Ambiguity: The paper demonstrates that by "injecting" label information into the PCA process, the model significantly reduces the overlap (ambiguity) between different tumor types in the reduced-dimensional space.
  • Cross-Disease Markers: The method successfully identified genes that are relevant across multiple cancer views. For example, while many methods found GNAS and KRT8, SDSPCA was uniquely able to identify PDGFRB and ABCC1—genes known in medical literature to be critical cancer markers but often missed by unsupervised dimensionality reduction.

Key Results

  • Classification Accuracy: SDSPCA achieved an average accuracy of 91.82%, outperforming:
  • Classical PCA (88.25%)
    • Linear Discriminant Analysis (LDA) (75.24%)
    • Other state-of-the-art sparse PCA variants like SPCArt (90.85%) and Z-SPCA (84.84%).
  • Biological Validation: Using KEGG pathway analysis, the authors showed that the genes selected by SDSPCA are heavily involved in "Focal adhesion" and "ECM-receptor interaction"—pathways known to be fundamental to how cancer cells spread and metastasize.
  • Computational Efficiency: Despite being an iterative supervised method, SDSPCA's computation time remained stable and comparable to standard PCA, making it practical for large-scale genomic datasets.

Limitations and Future Work

The authors acknowledge a few areas for improvement:

  • Linearity: SDSPCA is a linear model. Biological systems are often non-linear, so the model might miss complex geometric structures in the data.
  • Sensitivity to Outliers: The use of squared error (Frobenius norm) in the objective function can make the model sensitive to extreme outliers in the data.
  • Future Directions: The researchers suggest extending the model to capture non-linear structures and applying it to even more diverse biological datasets beyond tumor classification.

Summary updated

Abstract

Principal component analysis (PCA) has been used to study the pathogenesis of diseases. To enhance the interpretability of classical PCA, various improved PCA methods have been proposed to date. Among these, a typical method is the so-called sparse PCA, which focuses on seeking sparse loadings. However, the performance of these methods is still far from satisfactory due to their limitation of using unsupervised learning methods; moreover, the class ambiguity within the sample is high. To overcome this problem, this paper developed a new PCA method, which is named the supervised discriminative sparse PCA (SDSPCA). The main innovation of this method is the incorporation of discriminative information and sparsity into the PCA model. Specifically, in contrast to the traditional sparse PCA, which imposes sparsity on the loadings, here, sparse components are obtained to represent the data. Furthermore, via the linear transformation, the sparse components approximate the given label information. On the one hand, sparse components improve interpretability over the traditional PCA, while on the other hand, they are have discriminative abilities suitable for classification purposes. A simple algorithm is developed, and its convergence proof is provided. SDSPCA has been applied to the common-characteristic gene selection and tumor classification on multiview biological data. The sparsity and classification performance of SDSPCA are empirically verified via abundant, reasonable, and effective experiments, and the obtained results demonstrate that SDSPCA outperforms other state-of-the-art methods.
BibTeX
@article{feng2019supervised,
  author = {Chun-Mei Feng and Yong Xu and Jin-Xing Liu and Ying-Lian Gao and Chun-Hou Zheng},
  title = {{Supervised Discriminative Sparse PCA for Com-Characteristic Gene Selection and Tumor Classification on Multiview Biological Data}},
  journal = {IEEE Trans. Neural Networks Learn. Syst.},
  year = {2019},
  url = {https://openreview.net/forum?id=-bjDfmQL1V},
}

Loading related content...

Figure