Abstract
Large-scale chemical reaction datasets extracted from patent literature are essential for training predictive models but are often contaminated with noise, such as missing reagents or incorrect products. This paper introduces an automated "unassisted" noise reduction strategy that leverages the phenomenon of catastrophic forgetting in neural networks. By monitoring which reactions a model repeatedly learns and forgets during training, the authors identify and filter out chemically inconsistent data. This approach significantly improves the performance of retrosynthesis models, increasing round-trip accuracy by 13–15% without requiring manual rule-based filtering.
The Problem of Noisy Data in AI-Driven Chemistry
In modern computational chemistry, models learn to predict reactions by processing millions of examples from patent databases like USPTO. However, these datasets, generated via automated text-mining, are notoriously noisy. Common errors include missing reagents, swapped reactants, or chemically impossible products. Manual curation is prohibitively expensive, and traditional rule-based filters often inadvertently remove rare but valid chemical discoveries.
Methodology: The Forgotten Events Strategy
The researchers developed the "Forgotten Events Strategy," inspired by catastrophic forgetting—the tendency of deep learning models to lose information about previously learned tasks when exposed to new ones.
The process involves:
- Training & Tracking: A "Molecular Transformer" model—a sequence-to-sequence architecture that treats chemical reactions as a translation task—is trained on a large reaction dataset.
- Monitoring Forgetting: For every reaction, the model tracks whether it is predicted correctly at the end of each epoch.
- Hypothesis: Chemically sound reactions following established "grammars" of organic chemistry are retained by the model. Conversely, "noisy" or nonsensical reactions might be briefly memorized but are likely "forgotten" as the model internalizes broader, consistent chemical rules.
- Filtering: Reactions flagged with high forgetting frequencies are removed as noise.
Data and Definitions
The study utilized two primary data sources: Pistachio (2.4 million reactions) and USPTO (Lowe) (1 million reactions). To validate the strategy, "artificial noise" datasets were created by intentionally shuffling products or substituting them with incorrect predictions.
Key Results and Insights
- Retrosynthesis Performance: Round-trip accuracy—a metric where a second model verifies the first's suggestion—increased by 13 to 15 percentage points.
- Model Confidence: Cleaned models showed a 30% to 40% decrease in Cumulative Jensen-Shannon Divergence (CJSD), indicating higher certainty and better distribution quality.
- Noise Detection Accuracy: The protocol identified 99% of "easy" noise (shuffled data) and 60% of "subtle" noise (plausible but incorrect products).
- Efficiency: Removing up to 25% of the training data did not degrade forward-prediction performance; instead, it enhanced robustness by focusing on high-quality examples.
Limitations and Future Outlook
A fundamental limit exists around 30% data removal, where the model begins to lose meaningful, albeit rare, chemical knowledge. Future work may focus on "repopulating" datasets by using cleaned models to correct noisy entries or supplementing them with high-quality synthetic data to preserve rare functional groups.
Summary updated