ALMol: Aligned Language-Molecule Translation LLMs through Offline Preference Contrastive Optimisation

Dimitris GkoumasACL 2024

TL;DR

This paper introduces ALMol, a novel approach for training Large Language Models (LLMs) to translate between chemical language (like SMILES strings) and natural language descriptions. By using "contrastive preference optimization" on a small fraction of data, ALMol achieves significant improvements in translation accuracy and factual consistency, even outperforming models trained on much larger datasets. This work also proposes a new evaluation method to better assess "hallucinations" in LLMs within this domain.

What Problem Does This Paper Address?

The intersection of AI and chemistry holds immense potential for accelerating scientific discovery, particularly in areas like drug development and materials science. Large Language Models (LLMs) have shown promise in this domain, but effectively training them for tasks like translating between molecular representations (e.g., SMILES strings) and natural language descriptions remains challenging. Existing methods often rely on massive datasets and models, and can struggle with out-of-distribution data and generating translations that are merely "adequate" rather than truly accurate. Furthermore, evaluating the factual correctness and identifying "hallucinations" (inaccurate or nonsensical outputs) in these models is difficult. This paper aims to address these issues by developing a more efficient and robust training methodology and a better evaluation framework.

Methods

The core innovation of this paper is the Contrastive Preference Optimisation (CTO) method for training LLMs for language-molecule translation. CTO is inspired by Reinforcement Learning with Human Feedback (RLHF) but adapts it to work with offline preference data, meaning it doesn't require real-time human feedback during training. Instead, it uses pairs of preferred (human-annotated) and dis-preferred (model-generated) translations to guide the LLM.

Key aspects of the method include:

  • Offline Preference Data: CTO utilizes a dataset of (input, preferred output, dis-preferred output) triples.
  • Contrastive Loss: The training objective aims to maximize the probability of preferred outputs while minimizing the probability of dis-preferred ones, effectively learning from comparisons.
  • Reduced Data Requirement: A significant aspect is that CTO can achieve strong performance using only 10% of the available training data, addressing the need for massive datasets.
  • Behavior Cloning Regularizer: To ensure the model stays close to the desired data distribution and avoids generating arbitrary translations, a behavior cloning (BC) regularizer is added.
  • Proposed Evaluation Methodology: The authors introduce a fine-grained, domain-agnostic evaluation method that includes:
  • QAFactEval: For molecule-to-language translation, this metric assesses factual consistency by generating questions from the output caption and answering them using the reference text. It measures semantic overlap, F1 accuracy, and answerability.
  • Chr-F Metric: For language-to-molecule translation, this metric evaluates character n-gram matches between generated and reference molecules.
  • Bias Evaluation: Analysis of character and token length bias in generated outputs.

Datasets

The primary dataset used in this work is the L+M-24 benchmark dataset. This dataset comprises language-molecule pairs across four categories with applications in various scientific domains. The authors specifically utilize 10% of the training and validation subsets for their experiments. For creating the preference triples needed for CTO, they use MolT5 (Edwards et al., 2022) to generate dis-preferred outputs. For evaluation, they randomly select 3,000 unseen pairs from a distinct subset of the L+M-24 dataset.

Key Insights and Ideas

  • Efficiency through CTO: The most striking insight is that Contrastive Preference Optimisation (CTO) can achieve superior performance using a significantly smaller fraction of data (10%) compared to models trained on the entire dataset. This suggests that data efficiency is a critical factor in this domain.
  • Robustness to Out-of-Distribution Data: Models trained with CTO demonstrate remarkable robustness when tested on out-of-distribution data, outperforming baseline models that were trained on extensive datasets.
  • Language's Pivotal Role: The experiments suggest that language plays a more dominant role in molecule-related tasks than initially assumed. Models initialized with known cross-modal weights (CTO-Meditron-Mol) struggled with molecular patterns, while those initialized with agnostic cross-modals (CTO-Meditron-Lan) performed better, highlighting the importance of language understanding.
  • Evaluating Hallucinations is Crucial: The proposed QAFactEval metric provides a more nuanced way to assess factual consistency and identify hallucinations, moving beyond simple overlap metrics.

Key Results

  • Molecule-to-Language Translation: ALMol models achieved up to a 32% improvement in performance compared to Meditron (trained on the entire dataset) across various evaluation metrics. They also consistently outperformed the leading baseline, Meditron, even in agnostic cross-modal scenarios.
  • Language-to-Molecule Translation: While ALMol models showed improved performance compared to Meditron, they struggled to learn molecular patterns when initialized with agnostic cross-modals.
  • Factual Consistency: For molecule-to-language translation, ALMol (specifically CTO-Meditron-Mol) demonstrated superior factual consistency, achieving a semantic overlap of 2.08 (vs. 1.34 for Meditron), F1 accuracy of 0.34 (vs. 0.20), and answerability of 0.68 (vs. 0.51).
  • Length Bias: ALMol models did not exhibit significant length bias in their outputs, unlike Meditron which generated significantly shorter answers.

Limitations and Future Work

  • Language-to-Molecule Challenges: The authors acknowledge that their models struggled to learn molecular patterns effectively in the language-to-molecule task, particularly when initialized with agnostic cross-modals. They aim to explore more advanced initialization methods to address this.
  • Domain-Agnostic Evaluation: While the proposed evaluation method is domain-independent, further research could explore its applicability and effectiveness across an even wider range of scientific domains.
  • Generalizability of CTO: While CTO shows promise, further investigation into its generalizability across different LLM architectures and scientific modalities would be beneficial.

Summary updated

BibTeX
@inproceedings{gkoumas2024almol,
  author = {Dimitris Gkoumas},
  title = {{ALMol: Aligned Language-Molecule Translation LLMs through Offline Preference Contrastive Optimisation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure