Feature reduction for machine learning on molecular features: The GeneScore

Alexander Denker, Anastasia Steshina, Theresa Grooss, Frank Ueckert, Sylvia NürnbergarXiv 2021 · arXiv

Abstract

In precision oncology, the "curse of dimensionality" poses a significant challenge for machine learning models, as the number of molecular features often vastly exceeds the number of available patient samples. This paper introduces the GeneScore, a novel feature engineering method that collapses multiple complex molecular data types—including mutations, copy number variations, and gene expression—into a single continuous numerical value per gene. By integrating expert biological knowledge and weighting features by their deleterious impact, the GeneScore approach significantly improves classification accuracy in cancer diagnostics, outperforming traditional binary representation methods.

Introduction

The field of precision oncology is characterized by an abundance of data but a scarcity of observations. A single tumor profile can yield tens of thousands of features, yet clinical studies are typically limited to a few hundred patients. This imbalance often leads to overfitting, where models memorize training data rather than learning generalizable biological patterns. Furthermore, traditional machine learning approaches often rely on binary matrices (indicating the presence or absence of a mutation), which discard the biological "flavor" and intensity of molecular alterations. To address these issues, we propose a composite metric that preserves biological context while reducing dimensionality through feature integration.

Methods

The GeneScore is a composite metric that aggregates five distinct molecular data types into a single continuous score for every gene ($i$) in every patient ($j$):

$$Score(i,j) = SNV + Indel + CNV + Fusion + Expression$$

To ensure these disparate data types are additive, each is rescaled to a range (typically 0 to 1) based on its predicted biological impact:

  • SNVs (Single-Nucleotide Variants): Weighted using CADD (Combined Annotation Dependent Depletion) scores to predict deleteriousness.
  • Indels (Insertions/Deletions): Scored based on the presence of frameshifts (high damage) versus non-frameshift alterations (moderate damage).
  • CNV (Copy-Number Variants): Rescaled based on total copy number.
  • Fusion Genes: Represented as a binary 0 or 1.
  • Expression: Calculated as the $log_2$ fold change relative to the mean of healthy tissue from the GTEx database.
Conceptual framework of the GeneScore integration process.

The resulting GeneScore matrices were used to train an XGBoost classifier, which was then compared against a standard binary matrix baseline.

Results

The GeneScore approach demonstrated a substantial performance leap over binary matrices in a binary classification task (Sarcoma vs. other cancers). As shown in Table 1, the integrated score provided a much stronger signal for the model.

Table 1: Performance Comparison for Sarcoma Classification | Metric | GeneScore (No RFE) | Binary Matrix (No RFE) | | :--- | :--- | :--- | | Test Accuracy | 0.906 | 0.697 | | Test AUC | 0.965 | 0.710 | | F1 Score | 0.867 | 0.496 |

In a more complex five-way classification task involving different cancer types, the GeneScore maintained a high mean accuracy of 0.903. Analysis of the feature contributions revealed that gene expression is the dominant component of the GeneScore across most high-ranking features.

Relative contribution of molecular data types across top features.

Discussion and Conclusion

The primary insight of this work is that biological nuance matters. Moving from a binary "Is it broken?" assessment to a weighted "How broken is it?" metric provides machine learning models with the necessary signal to distinguish between cancer entities effectively. Gene expression emerged as the most significant predictor, suggesting it serves as the most immediate indicator of a cell's functional state.

While the GeneScore significantly improves accuracy, challenges remain regarding feature stability. The specific genes identified as most important varied across different model runs, indicating that the model has not yet converged on a definitive molecular signature. Future work will focus on refining weighting schemes and validating the method across larger, independent datasets like TCGA to ensure the universality of the scoring thresholds.

Summary updated

BibTeX
@misc{denker2021feature,
  author = {Alexander Denker and Anastasia Steshina and Theresa Grooss and Frank Ueckert and Sylvia Nürnberg},
  title = {{Feature reduction for machine learning on molecular features: The GeneScore}},
  year = {2021},
  eprint = {2101.05546},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2101.05546},
}
Figure