Efficient Machine Learning Approach for Yield Prediction in Chemical Reactions

Supratim Ghosh, Nupur Jain, Raghavan B. SunojarXiv 2025 · DOI · arXiv

Abstract

Developing machine learning (ML) models for yield prediction of chemical reactions has emerged as an important use case scenario in very recent years. In this space, reaction datasets present a range of challenges mostly stemming from imbalance and sparsity. Herein, we consider chemical language representations for reactions to tap into the potential of natural language processing models such as the ULMFiT (Universal Language Model Fine Tuning) for yield prediction, which is customized to work across such distribution settings. We contribute a new reaction dataset with more than 860 manually curated reactions collected from literature spanning over a decade, belonging to a family of catalytic meta-C(sp2)-H bond activation reactions of high contemporary importance. Taking cognizance of the dataset size, skewness toward the higher yields, and the sparse distribution characteristics, we developed a new (i) time- and resource-efficient pre-training strategy for downstream transfer learning, and (ii) the CFR (classification followed by regression) model that offers state-of-the-art yield predictions, surpassing conventional direct regression (DR) approaches. Instead of the prevailing pre-training practice of using a large number of unlabeled molecules (1.4 million) from the ChEMBL dataset, we first created a pre-training dataset SSP1 (0.11 million), by using a substructure-based mining from the PubChem database, which is found to be equally effective and more time-efficient in offering enhanced performance. The CFR model with the ULMFiT-SSP1 regressor achieved an impressive RMSE of 8.40 for the CFR-major and 6.48 for the CFR-minor class in yield prediction on the title reaction, with a class boundary of yield at 53 %. Furthermore, the CFR model is highly generalizable as evidenced by the significant improvement over the previous benchmark reaction datasets.

Introduction

Predicting the yield of a chemical reaction is a fundamental goal in synthetic chemistry, potentially eliminating thousands of hours of trial-and-error experimentation. However, machine learning models for this task face significant hurdles related to data quality. While High-Throughput Experimentation (HTE) datasets provide "clean" data, they often cover a very narrow chemical space. Conversely, data mined from literature is typically sparse, small (often under 1,000 entries), and heavily imbalanced due to "success bias," where failed reactions are rarely published. Standard Direct Regression (DR) models struggle to generalize across these skewed distributions.

Methodology: A Language-Based Approach

To address these challenges, researchers utilized a Natural Language Processing (NLP) approach, treating chemical reactions as sequences of information. Molecules are represented using SMILES strings and processed through the ULMFiT (Universal Language Model Fine Tuning) architecture, which employs a multi-layer Long Short-Term Memory (LSTM) network.

Overview of C-H functionalization and the application of NLP models in chemical reaction space.

The core innovations of this study are:

  1. Substructure-based Sampling (SSP): Rather than pre-training on millions of random molecules, the authors mined a targeted set of 110,000 molecules from PubChem (SSP1) that share specific structural features with the target reactions. This approach proved to be significantly more time-efficient.
  2. Classification Followed by Regression (CFR): Instead of a single regression model, a pipeline was developed. A classifier first determines if a reaction is likely to be "Major" (high yield) or "Minor" (low yield) based on a 53% yield boundary. Two specialized regression models then predict the final yield for each class, preventing the model from being "confused" by the lack of data in low-yield regions.

The m-CHA Dataset

A key contribution of this work is the m-CHA dataset, a manually curated collection of 866 reactions involving meta-C(sp2)–H bond activation. This catalytic process is of high contemporary importance for drug discovery and materials science.

Representative example and components of the meta-C(sp2)-H activation (m-CHA) reaction dataset.

The model was also validated against established benchmarks, including Buchwald-Hartwig (BH) coupling, Suzuki coupling (SC), and the massive USPTO dataset.

Key Results and Discussion

Superior Accuracy and Efficiency

On the m-CHA dataset, the CFR model achieved a Root Mean Squared Error (RMSE) of 8.40 for high-yield reactions and 6.48 for low-yield ones, significantly outperforming standard direct regression. The approach also reduced extreme outliers (predictions off by >20 units) from 5% to just 1-2%. Furthermore, the SSP1 pre-training strategy reduced training time from 23 hours to just 2.5 hours on an NVIDIA A100 GPU, demonstrating that data relevance is more critical than sheer volume.

Generalizability and Limitations

The CFR model set new state-of-the-art benchmarks for the Buchwald-Hartwig and Suzuki coupling datasets, proving its broad applicability. However, the authors noted a limitation with the asymmetric hydrogenation (AH) dataset, which is so extremely skewed that the CFR model performed slightly worse than direct regression.

Based on these findings, the authors propose a "rule of thumb": the CFR approach is most effective when the skewness ((\gamma)) of the yield distribution is between -1 and 1. For more extreme imbalances, traditional methods may still be preferable. Future research will explore refining classifiers for extreme imbalances and investigating transformer-based architectures.

Summary updated

Abstract

Developing machine learning (ML) models for yield prediction of chemical reactions has emerged as an important use case scenario in very recent years. In this space, reaction datasets present a range of challenges mostly stemming from imbalance and sparsity. Herein, we consider chemical language representations for reactions to tap into the potential of natural language processing models such as the ULMFiT (Universal Language Model Fine Tuning) for yield prediction, which is customized to work across such distribution settings. We contribute a new reaction dataset with more than 860 manually curated reactions collected from literature spanning over a decade, belonging to a family of catalytic meta-C(sp2)-H bond activation reactions of high contemporary importance. Taking cognizance of the dataset size, skewness toward the higher yields, and the sparse distribution characteristics, we developed a new (i) time- and resource-efficient pre-training strategy for downstream transfer learning, and (ii) the CFR (classification followed by regression) model that offers state-of-the-art yield predictions, surpassing conventional direct regression (DR) approaches. Instead of the prevailing pre-training practice of using a large number of unlabeled molecules (1.4 million) from the ChEMBL dataset, we first created a pre-training dataset SSP1 (0.11 million), by using a substructure-based mining from the PubChem database, which is found to be equally effective and more time-efficient in offering enhanced performance. The CFR model with the ULMFiT-SSP1 regressor achieved an impressive RMSE of 8.40 for the CFR-major and 6.48 for the CFR-minor class in yield prediction on the title reaction, with a class boundary of yield at 53 %. Furthermore, the CFR model is highly generalizable as evidenced by the significant improvement over the previous benchmark reaction datasets.
BibTeX
@misc{ghosh2025efficient,
  author = {Supratim Ghosh and Nupur Jain and Raghavan B. Sunoj},
  title = {{Efficient Machine Learning Approach for Yield Prediction in Chemical Reactions}},
  year = {2025},
  eprint = {2502.19976},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.19976},
  howpublished = {arXiv},
}
Figure