Abstract
Large Language Models (LLMs) have demonstrated remarkable capabilities in general reasoning but often struggle with the specialized domain of inorganic materials science. This paper introduces Mat-Instructions, the first large-scale instruction-tuning dataset specifically curated for this field, containing 349,090 entries. By fine-tuning LLaMA-7B on this dataset, researchers achieved substantial performance improvements in tasks such as crystal property prediction and chemical reaction forecasting, significantly outperforming both general-purpose and existing science-focused models.
Introduction
While models like GPT-4 and LLaMA excel at natural language tasks, they frequently "hallucinate" or fail when applied to materials science. The "language" of materials—comprising complex 3D crystal structures, symmetry operations, and interdisciplinary data—is fundamentally different from the text corpora used for general training. The authors identify three primary roadblocks: high annotation costs, domain complexity requiring specialized knowledge, and a lack of standardized frameworks for representing materials data (e.g., CIF vs. VASP files) in a format suitable for LLMs.
Methodology
The researchers developed a comprehensive five-step pipeline to transform raw scientific data into "instruction-response" pairs.
- Standardization: Diverse data formats were converted into a unified, concise text representation including lattice parameters and atomic coordinates.
- Instruction Generation: A "human-AI collaboration" approach was used to design diverse prompts. An LLM generated multiple variations of requests (e.g., "Determine the crystal lattice structure...") to ensure model flexibility.
- Fine-Tuning: The LLaMA-7B model served as the base and was fine-tuned across six core task categories: Crystal Structure Prediction (CSP), Property Prediction, Description-guided Design, Reaction Prediction, Retrosynthesis, and Crystal Description.
Dataset Composition
The Mat-Instructions dataset integrates data from major repositories such as the Materials Project, JDFT, MatKG, and MGED-KG. It comprises 349,090 instructions across several key scientific tasks.
The dataset distribution highlights a focus on Property Prediction (52.6%), followed by Structure Prediction and Design (29.2%), and Chemical Processes (16.1%). A smaller portion (2.1%) is dedicated to Description Generation, which translates structural data into human-readable summaries.
Key Results and Insights
The fine-tuned model, LLaMA-7B (FT), demonstrated dramatic improvements over base models and specialized alternatives like HoneyBee or Galactica:
- Reaction Prediction: Correctness rate improved to 49.73%, compared to 13.55% for the base model.
- Property Prediction: The Mean Absolute Error (MAE) dropped to a highly competitive 3.472.
- Retrosynthesis: The model reached a 42.06% correctness rate, whereas most general LLMs score near zero.
A key insight is that 3D structural information can be effectively "tokenized" into text strings for LLM reasoning, provided the formatting is consistent. Furthermore, "instruction tuning" proved to be a highly efficient method for injecting scientific expertise into pre-trained models compared to training from scratch.
Limitations and Future Directions
Despite these gains, challenges remain. In Crystal Structure Prediction (CSP), while the model's "Valid rate" (generating physically sensible structures) was 20%, the "Match rate" (exact ground-truth match) was only 1.29%, highlighting the difficulty of 3D spatial reasoning for current architectures. Additionally, the dataset is currently skewed toward crystals with fewer than 10 atoms. Future work aims to tackle larger-scale materials and integrate multimodal data, such as crystal diagrams and experimental videos, to create a more holistic Materials LLM.
Code and dataset are available at: https://github.com/zjuKeLiu/Mat-Instructions
Summary updated