Training a Scientific Reasoning Model for Chemistry

Siddharth Narayanan, James D. Braza, Ryan-Rhys Griffiths, Albert Bou, Geemi P. Wellawatte, Mayk Caldas Ramos, Ludovico Mitchener, Samuel G. Rodriques, Andrew D. White

TL;DR

This paper introduces "ether0," a 24B parameter LLM trained to perform scientific reasoning in chemistry. By leveraging reinforcement learning on a large, experimentally-grounded dataset, ether0 significantly outperforms existing models and human experts on various molecular design tasks, demonstrating that reasoning capabilities can be effectively transferred to scientific domains beyond math and programming.

What Problem Does This Paper Address?

Large language models (LLMs) have shown impressive capabilities, but their reasoning abilities have largely been confined to domains like mathematics, programming, and logic. A key challenge is extending this reasoning power to complex scientific fields, such as chemistry, where problems often involve intricate, multi-step processes and require generating novel solutions rather than just retrieving information. The authors aim to bridge this gap by developing a reasoning model specifically for chemistry tasks, demonstrating that such models can be trained efficiently and achieve state-of-the-art performance.

Methods

The core of the paper is the development of "ether0," a 24B parameter LLM based on Mistral-Small-24B-Instruct. The training pipeline involves several stages:

  • Supervised Fine-Tuning (SFT): Initial training on long chain-of-thought reasoning sequences to provide a strong starting point for reinforcement learning.
  • Task-Specific Reinforcement Learning (RL): Group Relative Policy Optimization (GRPO) is used to train specialized "expert" models for different categories of chemistry tasks. This stage focuses on learning from verifiable rewards.
  • Distillation: The knowledge from the specialized models is distilled into a single, "generalist" model.
  • All-Task Reinforcement Learning (RL): The generalist model undergoes further GRPO training across all tasks to consolidate its capabilities.
  • Safety Alignment: A final RL phase is employed to ensure the model refuses to engage with unsafe chemistry tasks.

A key innovation is the use of reinforcement learning with verifiable rewards, allowing the model to explore and learn from its own generated outputs, overcoming limitations of fixed datasets. They also introduce an "advantage-based curriculum" to improve RL efficiency.

Datasets

The authors constructed a novel dataset of 640,730 experimentally-grounded chemistry problems spanning 375 tasks. These tasks cover a wide range of chemical reasoning, including:

  • Molecular Design & Synthesis: Synthesizability, retrosynthesis, reaction prediction, molecule completion, molecular formula, functional group assignment.
  • Property Prediction: Blood-brain barrier permeability, receptor binding, ADME properties, aqueous solubility, LD50, pKa, photoswitches.
  • Structure Elucidation: IUPAC name generation, molecular captioning.
  • Safety & Scent: Predicting safety-related properties and olfactory attributes.

This dataset is unique in its reliance on experimentally verified data, distinguishing it from benchmarks that might rely on hypothetical molecules or purely knowledge-based questions.

Key Insights and Ideas

  • Reasoning Generalizes to Science: The success of ether0 demonstrates that LLM reasoning capabilities, honed in domains like math, can be effectively transferred to complex scientific fields like chemistry.
  • RL is Key for Scientific Reasoning: Reinforcement learning, particularly with verifiable rewards, is crucial for enabling LLMs to learn intricate scientific reasoning processes and discover novel solutions that might not be present in static datasets.
  • Data Efficiency is Achievable: By using RL, ether0 achieves competitive or superior performance with significantly less data compared to traditional, domain-specific models.
  • Cognitive Behaviors Emerge Selectively: The emergence of reasoning behaviors like backtracking and verification is not uniform across all tasks but is selectively amplified in tasks where structured reasoning is most advantageous.
  • Safety Alignment is Crucial: A dedicated safety alignment phase is necessary to prevent LLMs from generating harmful or unsafe chemical information.

Key Results

  • State-of-the-Art Performance: ether0 outperforms general-purpose LLMs, frontier chemistry models, and even human experts on many open-answer chemistry tasks. For example, it achieves higher accuracy than the Molecular Transformer (MT) on reaction prediction with significantly less training data.
  • Data Efficiency: ether0 demonstrates superior data efficiency, achieving high accuracy with fewer training examples compared to other models.
  • Improved Reasoning Quality: Filtering out "bad" reasoning during training leads to marginally higher quality reasoning and boosts performance.
  • Human Expert Comparison: Human experts (chemists) achieved an average accuracy of around 30-40% on several tasks, while ether0 achieved significantly higher scores.

Limitations and Future Work

  • Generalization Beyond Training Distribution: ether0's performance might be limited on domains outside its training distribution, such as inorganic chemistry.
  • Instruction Following Degradation: Intensive RL training may slightly degrade general instruction-following and chat capabilities.
  • Tool Integration: The model does not currently integrate with external tools, which are often crucial in real-world chemistry workflows.
  • Future Work: The authors suggest integrating chemistry reasoning with tool-calling capabilities into a single model. They also anticipate this approach can be applied to other scientific domains.

Summary updated

Abstract

Reasoning models are large language models that emit a long chain-of-thought before answering, providing both higher accuracy and explicit reasoning for their response. A major question has been whether language model reasoning generalizes beyond mathematics, programming, and logic, where most previous work has focused. We demonstrate that reasoning models can be post-trained for chemistry without additional domain pretraining, and require substantially less data compared to contemporary domain-specific models. We report ether0, a 24B parameter LLM (based on Mistral-Small-24B) that can reason in natural language and respond with chemical structures. This reasoning model was trained with reinforcement learning on 640,730 experimentally-grounded chemistry problems across 375 tasks ranging from synthesizability, to blood-brain barrier permeability, to human receptor activity, to scent. Our model exceeds general-purpose chemistry models, frontier models, and human experts on molecular design tasks. It is also more data efficient relative to specialized models. We anticipate that this method can be applied to train data-efficient language models specialized for tasks across a wide variety of scientific domains.
BibTeX
@inproceedings{narayanan2025training,
  author = {Siddharth Narayanan and James D. Braza and Ryan-Rhys Griffiths and Albert Bou and Geemi P. Wellawatte and Mayk Caldas Ramos and Ludovico Mitchener and Samuel G. Rodriques and Andrew D. White},
  title = {{Training a Scientific Reasoning Model for Chemistry}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=Mr5Pyb3MLk},
}
Figure