Author Correction: Small Molecule Accurate Recognition Technology (SMART) to Enhance Natural Products Research

Chen Zhang, Yerlan Idelbayev, Nicholas Roberts, Yiwen Tao, Yashwanth Nannapaneni, Brendan M. Duggan, Jie Min, Eugene C. Lin, Erik Gerwick, Garrison W. Cottrell, William H. GerwickScientific 2020 · DOI · OpenAlex

Abstract

This formal correction addresses a critical bug identified in the evaluation code of the 2017 publication introducing Small Molecule Accurate Recognition Technology (SMART). The error affected the calculation of precision-recall curves and associated Area Under the Curve (AUC) metrics. By re-evaluating the system using standard machine learning libraries, the authors provide a corrected and reliable performance benchmark for the natural products research community.

Introduction

The original research introduced SMART, a tool designed to automate the identification of small molecules in complex natural product mixtures. Identifying these molecules is a notoriously difficult task due to the chemical diversity and complexity of biological extracts. While the core deep learning architecture of SMART remains unchanged, this update rectifies the performance metrics reported in the initial study. The original precision-recall (PR) curves were generated using a flawed custom script, necessitating a re-evaluation to ensure the scientific record reflects the true performance of the technology.

Methodology

The correction focuses on the evaluation methodology used to assess the model's ability to identify correct molecules without excessive false positives.

  • Identification of Error: The original custom code used an incorrect relationship between recall and precision points, leading to skewed visualizations and inaccurate AUC values.
  • Standardized Re-evaluation: To ensure accuracy, reproducibility, and comparability, the authors recomputed all PR curves and AUC metrics using the scikit-learn library. This transition to industry-standard tools ensures that the performance data follows established machine learning definitions.

Results and Discussion

The corrected analysis provides updated performance data for two primary benchmarks: the SMART5 and SMART10 datasets.

Corrected Precision-Recall curves for SMART5 and SMART10 datasets, comparing various embedding dimensions (dim=2 to dim=10) and PCA baselines.

Impact of Embedding Dimensionality

The corrected figures (Panels a-d) confirm that increasing the dimensionality of molecular embeddings significantly enhances recognition performance. For the SMART10 dataset, the system's AUC improved from 0.3861 at dim=2 to 0.5287 at dim=10. Similarly, the SMART5 dataset showed an AUC of 0.1445 at the highest dimensionality tested. These results reinforce the idea that higher-dimensional representations capture more nuanced molecular features.

SMART vs. Traditional Baselines

A key finding of the SMART project is its superiority over traditional linear dimensionality reduction techniques. The corrected "PCA Only" plots (Panels e-f) demonstrate this clearly, showing very low AUC values (0.0130 for SMART5 and 0.0532 for SMART10). Even with the corrected math, the SMART deep learning approach provides approximately a 10-fold improvement in recognition accuracy over the PCA baseline for the larger SMART10 dataset.

Conclusion

This correction ensures that the performance metrics for the SMART system are accurate and reliable. By adopting standardized evaluation tools, the authors have reinforced the integrity of the SMART project's data, confirming its significant performance gains over traditional analytical methods and providing a solid foundation for future natural products research.

Summary updated

Abstract

An amendment to this paper has been published and can be accessed via a link at the top of the paper.
BibTeX
@inproceedings{zhang2020author,
  author = {Chen Zhang and Yerlan Idelbayev and Nicholas Roberts and Yiwen Tao and Yashwanth Nannapaneni and Brendan M. Duggan and Jie Min and Eugene C. Lin and Erik Gerwick and Garrison W. Cottrell and William H. Gerwick},
  title = {{Author Correction: Small Molecule Accurate Recognition Technology (SMART) to Enhance Natural Products Research}},
  booktitle = {Scientific},
  year = {2020},
  doi = {10.1038/s41598-020-60108-0},
}

Loading related content...

Figure