Predicting the Activity of Unidentified Chemicals in Complementary Bioassays from the HRMS Data to Pinpoint Potential Endocrine Disruptors

Ida Rahu, Meelis Kull, Anneli KruveJournal of Chemical Information and Modeling 2024 · DOI

TL;DR

This paper presents a novel approach to predict the endocrine-disrupting activity of chemicals directly from their mass spectrometry data, even when their structures are unknown. By leveraging probabilistic molecular fingerprints derived from HRMS data, the study demonstrates that machine learning models can effectively identify potential endocrine disruptors, significantly reducing the workload for analyzing complex environmental mixtures.

What Problem Does This Paper Address?

Identifying and assessing the toxicity of chemicals in the environment is a major challenge. While analytical techniques like liquid chromatography-high-resolution mass spectrometry (LC-HRMS) can detect thousands of chemicals in complex mixtures, a vast majority remain structurally unidentified. This lack of identification hinders our ability to assess their potential health risks, particularly for endocrine disruptors (EDs). Traditional toxicity testing is time-consuming and resource-intensive, and existing in silico methods often require known chemical structures, which are unavailable for many detected compounds. This paper tackles the critical gap of predicting chemical activity from HRMS data without prior structural identification, aiming to prioritize chemicals for further investigation.

Methods

The core of this research lies in bridging the gap between HRMS data and predictive toxicology. The authors employed a workflow that starts with cleaning and preparing the Tox21 dataset. A key innovation is the use of SIRIUS+CSI:FingerID to generate probabilistic molecular fingerprint features directly from HRMS data. Unlike traditional binary fingerprints, these probabilistic features represent the likelihood of a structural pattern being present.

To handle these probabilistic inputs, the researchers developed a Monte Carlo sampling strategy. Instead of using a fixed threshold (e.g., 0.5) to convert probabilities to binary values, this method samples the probabilistic fingerprint features multiple times (N=10,000) to create multiple binary representations. These are then fed into machine learning models, and the final prediction is an average of the outputs. This approach accounts for the uncertainty inherent in probabilistic features and leads to more robust predictions.

Both single-output models (one model per bioassay) and multi-output models (one model for all bioassays) were trained using various machine learning algorithms, including ensemble methods and deep neural networks. Model selection and evaluation were primarily based on the False Positive Rate at 90% Recall (FPRTPR = 0.9), a metric that balances the need for high detection rates with minimizing false alarms, crucial for reducing postprocessing efforts.

Datasets

The study utilized the Tox21 10K dataset, a collection of approximately 10,000 licensed drugs and environmental chemicals with associated activity data across 12 bioassays related to endocrine disruption.

  • Original Dataset: After initial cleaning and deduplication, this dataset contained 7483 unique chemicals.
  • Training Set: 5388 chemicals were used for training the models.
  • Intermediate Test Set: 1347 chemicals were used for model selection and hyperparameter tuning.
  • Real-Life Test Set: A subset of 734 chemicals, carefully sampled to match the original dataset's label proportions, was used for the final evaluation. This set was derived from experimental HRMS data obtained from MassBank and MassBank of North America (MONA) databases.

Key Insights and Ideas

  • HRMS Data as a Predictive Source: The study strongly validates the idea that HRMS data, even without full structural identification, contains sufficient information to predict chemical activity. This opens up new avenues for screening environmental samples.
  • Probabilistic Fingerprints and Monte Carlo Sampling: The probabilistic nature of fingerprints derived from HRMS is a challenge. The Monte Carlo sampling strategy effectively addresses this by creating multiple binary representations, leading to more stable and reliable predictions.
  • Single-Output vs. Multi-Output Models: While single-output models generally performed slightly better in terms of FPRTPR=0.9 for individual bioassays, the study acknowledges the potential of multi-output models, especially in leveraging correlations between bioassays, which could be beneficial for complex toxicological assessments.
  • Feature Importance and Toxicophores: SHAP analysis revealed that the predictive models are indeed learning characteristic structural patterns (toxicophores) associated with endocrine-disrupting activity, providing mechanistic insights.
  • Reduced Postprocessing Burden: The developed models, particularly when achieving a low FPRTPR=0.9, can significantly reduce the manual effort required to analyze HRMS data by prioritizing chemicals for further investigation.

Key Results

  • The models achieved a low false-positive rate at 90% recall (FPRTPR = 0.9) across 12 bioassays, ranging from 0.251 (sr.mmp) to 0.850 (nr.er) on the real-life test set.
  • This performance translates to a potential reduction in postprocessing workload by up to 75% for nontarget HRMS analysis.
  • The models demonstrated that fingerprint features derived from HRMS data are informative enough to predict endocrine-disrupting activity.
  • The Monte Carlo sampling strategy was shown to be crucial for consistent predictions when using probabilistic fingerprint features, with an average of only a small percentage of chemicals showing changed activity predictions compared to a naive approach.
  • The study observed similar trends in model performance and challenges (e.g., difficulty in predicting nr.er activity) compared to the Tox21 Data Challenge, despite using a more constrained feature set.

Limitations and Future Work

  • Dataset Coverage: The Tox21 dataset, while extensive, may not fully represent the chemical diversity found in all real-world environmental samples. This could limit the generalization capability of the trained models to novel chemical classes.
  • Feature Predefinition: The study relied on predefined fingerprint features extractable by SIRIUS+CSI:FingerID. Future work could explore additional information extractable from HRMS data or investigate other feature selection strategies.
  • Probabilistic Feature Accuracy: While SIRIUS+CSI:FingerID is accurate, high posterior probabilities do not guarantee the absolute presence of a feature. Inaccuracies, especially for complex or less well-represented chemical groups, could lead to misclassifications.
  • Data Quality: The study highlights the critical importance of high-quality, data-rich fragmentation spectra for accurate fingerprint prediction. Experimental methodologies that can assess and enhance spectral quality in real-time are needed.
  • Multi-output Model Optimization: While promising, the multi-output models did not consistently outperform single-output models in this study. Further research into their architecture and selection strategies is warranted.

The authors suggest that future work should focus on expanding the chemical space covered by the training data, exploring alternative feature extraction methods from HRMS, and further validating the proposed methodology and selection strategies.

Summary updated

BibTeX
@article{rahu2024predicting,
  author = {Ida Rahu and Meelis Kull and Anneli Kruve},
  title = {{Predicting the Activity of Unidentified Chemicals in Complementary Bioassays from the HRMS Data to Pinpoint Potential Endocrine Disruptors}},
  journal = {Journal of Chemical Information and Modeling},
  year = {2024},
  doi = {10.1021/acs.jcim.3c02050},
}
Figure