polyBART: A Chemical Linguist for Polymer Property Prediction and Generative Design

Anagha Savit, Harikrishna Sahu, Shivank S. Shukla, Wei Xiong, Rampi RamprasadEMNLP 2025

TL;DR

Researchers have developed polyBART, a novel language model that treats polymers like words in a language, enabling rapid and accurate prediction of their properties and the generation of entirely new polymer structures. This work introduces a new polymer representation (PSELFIES) that allows existing molecular language models to be adapted for polymers, paving the way for accelerated discovery of materials with desired characteristics.

What Problem Does This Paper Address?

Designing new polymers with specific properties is a monumental task due to the sheer vastness of the chemical space polymers occupy. While machine learning has made strides in predicting properties from known polymer structures (the "forward problem"), designing polymers to achieve desired properties (the "inverse problem") has been significantly harder. This is largely because traditional machine learning methods struggle to generate chemically valid polymer structures. Existing molecular language models, successful in small-molecule discovery, haven't been directly applicable to the unique complexities of polymer structures.

Methods

The core innovation is polyBART, a language model built upon the BART architecture, specifically adapted for polymers. To bridge the gap between existing molecular language models and polymers, the authors introduce Pseudo-polymer SELFIES (PSELFIES). PSELFIES is a novel string representation that extends the SELFIES (Self-Referencing Embedded Strings) format, which guarantees chemically valid molecular strings. PSELFIES allows polymers to be represented in a way that mirrors molecular syntax, making them compatible with existing language models.

polyBART is developed through continued pretraining of a molecular language model (SELFIES-TED) on a large dataset of PSELFIES strings. This process allows the model to learn the "grammar" and "syntax" of polymers. For property prediction, polyBART's encoder generates embeddings, which are then used to train a Gaussian Process Regressor (GPR). For generative design, the decoder part of polyBART generates new PSELFIES strings, which are then filtered for validity and desired properties.

Datasets

The authors created a large dataset of approximately 210 million PSELFIES strings for polymers. This dataset includes both known polymers and hypothetical ones generated using various polymerization reactions and popular click chemistry methods, ensuring synthetic feasibility. This dataset is used for pretraining polyBART.

For evaluation, they used datasets for thermal properties (glass transition temperature Tg, thermal degradation temperature Td, melting temperature Tm) and electronic properties (linear chain bandgap Eg, bulk polymer bandgap Egb, electron affinity Eea, ionization energy Ei).

Key Insights and Ideas

  • Polymers as a Language: The paper successfully frames polymer structures as a "language," enabling the application of powerful NLP techniques to polymer science.
  • PSELFIES as a Universal Translator: The PSELFIES representation is a crucial enabler, acting as a bridge that allows molecular language models to understand and generate polymer structures.
  • Bidirectional Capability: Unlike many previous models, polyBART can perform both property prediction and de novo polymer generation, offering a complete solution for polymer design.
  • Transfer Learning is Key: Initializing polyBART with a pretrained molecular model (SELFIES-TED) and then fine-tuning it on PSELFIES significantly outperforms training from scratch, highlighting the power of transfer learning in this domain.
  • Real-World Validation: The successful synthesis and experimental validation of a polymer predicted by polyBART for high thermal properties is a critical step, demonstrating the practical utility of the model.

Key Results

  • Property Prediction: polyBART achieves state-of-the-art results in predicting thermal properties, performing comparably to or better than existing models like polyBERT. It also shows strong performance against LLaMA-3 on thermal property prediction.
  • Generative Design: polyBART generates valid polymer structures with high success rates (100% molecule validity, 91-98% polymer validity) and high novelty.
  • Property-Conditioned Generation: The model can generate novel polymers that meet specific target properties (e.g., high Tg, high Eg) and are predicted to be synthetically accessible.
  • Experimental Validation: A polymer predicted by polyBART to have a high glass transition temperature (Tg) was synthesized and experimentally tested, showing that its measured Tg closely matched the model's prediction.

Limitations and Future Work

  • Generalizability of PSELFIES: The paper acknowledges that the generalizability of PSELFIES to other molecular foundation models and architectures needs further investigation.
  • Scope of Properties: While polyBART excels at thermal and electronic properties, its applicability to a broader range of polymer properties and simultaneous optimization of multiple properties requires further exploration.
  • Homopolymers Only: Currently, polyBART is limited to homopolymers and does not support the representation or generation of copolymers, polymer blends, or composites. Future work will focus on extending these capabilities.

Summary updated

BibTeX
@inproceedings{savit2025polybart,
  author = {Anagha Savit and Harikrishna Sahu and Shivank S. Shukla and Wei Xiong and Rampi Ramprasad},
  title = {{polyBART: A Chemical Linguist for Polymer Property Prediction and Generative Design}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure