Machine Learning-based Biomarkers Identification and Validation from Toxicogenomics - Bridging to Regulatory Relevant Phenotypic Endpoints

Sheikh Mokhlesur Rahman, Jiaqi Lan, David Kaeli, Jennifer Dy, Akram N. Alshawabkeh, April Z. GubioRxiv 2020 · DOI

TL;DR

This paper presents a machine learning approach to identify a minimal set of protein biomarkers from in vitro toxicogenomic data that can accurately predict in vivo toxicity endpoints like carcinogenicity and genotoxicity. By combining Maximum Relevance Minimum Redundancy (MRMR) feature selection with Support Vector Machines (SVM), they demonstrate that a small, well-chosen set of biomarkers can achieve high prediction accuracy, paving the way for more cost-effective and efficient toxicity screening.

What Problem Does This Paper Address?

Assessing the health and environmental risks of chemical exposure is crucial, and high-throughput in vitro assays combined with Adverse Outcome Pathway (AOP) frameworks show great promise. However, a major hurdle is establishing a reliable quantitative link between the molecular-level responses observed in in vitro assays and the actual phenotypic toxicity observed in living organisms (in vivo). Current toxicogenomics approaches often rely on a large number of potentially redundant markers, making the process expensive, time-consuming, and complex. This paper aims to bridge this gap by developing a method to identify a concise set of predictive biomarkers from in vitro data that can reliably forecast in vivo toxicity.

Methods

The core of this study lies in a two-step machine learning pipeline:

  1. Feature Selection (Biomarker Identification): The authors employed the Maximum Relevance Minimum Redundancy (MRMR) algorithm. This algorithm is designed to select features (in this case, protein biomarkers) that are highly relevant to the target outcome (e.g., carcinogenicity) while simultaneously minimizing redundancy among the selected features. They explored three scoring criteria within MRMR:
  2. t-stat: Measures the difference in protein expression between positive and negative samples (maximum relevance).
  3. MRMR-TCD: Combines t-stat with average correlation (redundancy) using subtraction.
  4. MRMR-TCQ: Combines t-stat with average correlation using division, applying a stronger penalty for redundancy.
  5. Classification (Prediction Model): The selected top-ranked biomarkers were then used to train a Support Vector Machine (SVM) classifier. SVMs were chosen for their ability to handle high-dimensional data and avoid overfitting, which is common in toxicogenomics. The performance of the SVM models was evaluated using 10-fold cross-validation.

Datasets

The study utilized an in vitro toxicogenomic assay dataset generated from exposing Saccharomyces cerevisiae (yeast) strains to 20 different chemicals. These chemicals included known genotoxic compounds and negative controls. The assay measured the real-time proteomic response of 38 GFP-fused proteins involved in DNA damage and repair pathways. The phenotypic endpoints for validation were:

  • In vivo rodent carcinogenicity: Data collected from existing literature.
  • Ames genotoxicity assay: Data collected from existing literature.

The paper does not introduce a new benchmark dataset but rather uses existing data to validate their biomarker discovery approach.

Key Insights and Ideas

  • The Power of Parsimony: The study strongly suggests that a small, carefully selected set of biomarkers can be as, or even more, effective than using a large panel of proteins for predicting toxicity. This is a crucial insight for making toxicogenomics more practical and cost-effective.
  • MRMR's Advantage: The MRMR algorithm, particularly the MRMR-TCQ variant, proved effective in identifying biomarkers that are both relevant and non-redundant, leading to more robust predictive models.
  • Divergent Biomarkers for Different Endpoints: Interestingly, the top-ranked biomarkers identified for predicting in vivo carcinogenicity were different from those identified for Ames genotoxicity. This highlights that different molecular pathways are implicated in these distinct toxicological outcomes.
  • Pathway-Specific Biomarkers: The identified biomarkers for carcinogenicity were heavily involved in double-strand break repair and DNA recombination, while those for Ames genotoxicity were linked to base- and nucleotide-excision repair. This aligns with our understanding of DNA damage response pathways.
  • Bridging In Vitro to In Vivo: The research successfully demonstrates a method to bridge the gap between molecular responses in vitro and phenotypic outcomes in vivo, a critical step towards realizing the goals of initiatives like Tox21.

Key Results

  • Carcinogenicity Prediction: The top five biomarkers achieved an AUC of 0.81 for predicting in vivo carcinogenicity. Using all 38 proteins increased the AUC to 0.88, but the top five provided a significant portion of the predictive power with much less complexity. The top five biomarkers for carcinogenicity were NTG2, RAD34, RAD27, MSH2, and YKU70.
  • Ames Genotoxicity Prediction: The top five biomarkers achieved an AUC of 0.75 for Ames genotoxicity prediction. Using all 38 proteins improved the AUC to 0.84. The top five biomarkers for Ames genotoxicity were APN2, RFA2, NTG2, RAD2, and MSH6.
  • Accuracy: For carcinogenicity, the top five biomarkers achieved ~76% accuracy, while using all proteins reached 83%. For Ames genotoxicity, the top five achieved 70% accuracy, increasing to 78% with all proteins.
  • Robustness: The high cross-validated accuracies suggest that the developed models are robust and generalize well.

Limitations and Future Work

  • Limited Chemical Set: The study used a relatively small set of 20 chemicals. Future work could involve validating the identified biomarkers on a larger and more diverse set of chemicals.
  • Yeast Model: While yeast is a valuable model for studying DNA repair pathways, its direct translation to mammalian systems requires further investigation. The authors acknowledge that the identified biomarkers are conserved across eukaryotes, suggesting potential applicability.
  • Temporal Dynamics: While the study uses time-series data, further exploration into the temporal dynamics of protein responses could yield deeper insights.
  • Expanding the AOP Framework: The authors suggest that their method can contribute to filling the knowledge gap in phenotypic anchoring and predictive toxicology, supporting the broader implementation of AOPs for risk assessment. Future work could focus on integrating these findings into more comprehensive AOP frameworks.

Summary updated

Abstract

ABSTRACT High-throughput in vitro assays and AOP-based approach is promising for the assessment of health and ecotoxicological risks from exposure to pollutants and their mixtures. However, one of the major challenges in realization and implementations of the Tox21 vision is the urgent need to establish quantitative link between in-vitro assay molecular endpoint and in-vivo phenotypic toxicity endpoint. Here, we demonstrated that, using time series toxicomics in-vitro assay along with machine learning-based feature selection (MRMR) and classification method (SVM), an “optimal” number of biomarkers with minimum redundancy can be identified for prediction of phenotypic endpoints with good accuracy. We included two case studies for in-vivo carcinogenicity and Ames genotoxicity prediction with 20 selected chemicals including model genotoxic chemicals and negative controls, respectively, using an in-vitro toxicogenomic assay that captures real-time proteomic response data of 38 GFP-fused proteins of S. cerevisiae strains covering biomarkers indicative of all known DNA damage and repair pathways in yeast. The results suggested that, employing the adverse outcome pathway (AOP) concept, molecular endpoints based on a relatively small number of properly selected biomarker-ensemble involved in the conserved DNA-damage and repair pathways among eukaryotes, were able to predict both in-vivo carcinogenicity in rats and Ames genotoxicity endpoints. The specific biomarkers identified are different for the two different phenotypic genotoxicity assays. The top-ranked five biomarkers for the in-vivo carcinogenicity prediction mainly focused on double strand break repair and DNA recombination, whereas the selected top-ranked biomarkers for Ames genotoxicity prediction are associated with base- and nucleotide-excision repair. Current toxicomics approach still mostly rely on large number of redundant markers without pre-selection or ranking, therefore, selection of relevant biomarkers with minimal redundancy would reduce the number of markers to be monitored and reduce the cost, time, and complexity of the toxicity screening and risk monitoring. The method developed in this study will help to fill in the knowledge gap in phenotypic anchoring and predictive toxicology, and contribute to the progress in the implementation of tox 21 vision for environmental and health applications. TOC Art
BibTeX
@misc{rahman2020machine,
  author = {Sheikh Mokhlesur Rahman and Jiaqi Lan and David Kaeli and Jennifer Dy and Akram N. Alshawabkeh and April Z. Gu},
  title = {{Machine Learning-based Biomarkers Identification and Validation from Toxicogenomics - Bridging to Regulatory Relevant Phenotypic Endpoints}},
  year = {2020},
  doi = {10.1101/2020.12.18.423486},
  howpublished = {bioRxiv},
}
Figure