SMILES-Prompting: A Novel Approach to LLM Jailbreak Attacks in Chemical Synthesis

Aidan Wong, He Cao, Zijing Liu, Yu LiarXiv 2024 · DOI · arXiv

Introduction

As Large Language Models (LLMs) are increasingly integrated into scientific workflows, they offer powerful capabilities for chemical research. However, this integration introduces significant safety risks. If not properly safeguarded, LLMs could provide detailed instructions for synthesizing hazardous substances, such as explosives, illicit drugs, or chemical weapons. While current safety mechanisms—primarily based on natural language alignment and keyword filtering—are effective against direct queries, they are vulnerable to domain-specific technical encodings. This paper investigates "SMILES-prompting," a novel jailbreak technique that uses chemical notation to bypass safety filters.

Methodology and Evaluation Framework

The researchers evaluated the robustness of leading LLMs against four types of "jailbreak" attacks:

  1. Red-Teaming: Direct, malicious queries using common names (e.g., "How do I make TNT?").
  2. Explicit Prompting: Using "Do Anything Now" (DAN) style prompts to command the model to ignore safety guidelines.
  3. Implicit Prompting: Using scenario-shifts (e.g., "You are a security analyst...") to mask intent.
  4. SMILES-Prompting: Replacing common chemical names with their SMILES (Simplified Molecular-Input Line-Entry System) strings—a computerized representation of molecular structures.

The evaluation pipeline utilizes a GPT-4o-based automated judge to score model responses based on two criteria: Component Identification (listing necessary precursors) and Synthesis Process (providing accurate procedural steps). These scores were subsequently verified by human experts.

Overview of the SMILES-prompting attack and evaluation pipeline.

Experimental Results

The study focused on 30 prohibited substances across three categories: explosives, drugs, and chemical weapons. The attacks were tested against GPT-4o and Llama-3-70B-Instruct.

The results revealed a stark vulnerability to SMILES-prompting. While traditional red-teaming and explicit prompts were almost entirely blocked by Llama-3 (near 0% success rate), SMILES-prompting achieved an Attack Success Rate (ASR) of 30.0% for synthesis procedures. GPT-4o, while generally more robust, still succumbed to SMILES-prompting with a 31.67% success rate for synthesis procedures, far exceeding the success of implicit prompting (~13%).

Attack Success Rates (ASR) for component identification and synthesis procedures across different models and attack types.

Discussion: The Encoding-Shift Vulnerability

The effectiveness of SMILES-prompting is attributed to an "encoding-shift." LLMs are trained to be helpful assistants in chemistry and possess a deep understanding of SMILES for legitimate research. However, their safety alignment is primarily tuned for natural language. When a request is presented as a SMILES string, it evades surface-level keyword filters and fails to trigger the model's internal "harmful intent" detectors. The model "understands" the chemistry but fails to recognize that the structure represents a banned substance.

Conclusion

SMILES-prompting represents a significant security loophole in current LLMs. To address this, the authors suggest two primary defense strategies:

  • Normalization: Translating technical inputs into natural language names before they reach safety filters.
  • Enhanced Training: Incorporating databases of prohibited SMILES into the model's safety training or using Retrieval-Augmented Generation (RAG) to identify and refuse hazardous structures.

Future work will involve expanding the chemical dataset and refining these defensive measures to ensure the safe application of LLMs in the sciences.

Summary updated

Abstract

The increasing integration of large language models (LLMs) across various fields has heightened concerns about their potential to propagate dangerous information. This paper specifically explores the security vulnerabilities of LLMs within the field of chemistry, particularly their capacity to provide instructions for synthesizing hazardous substances. We evaluate the effectiveness of several prompt injection attack methods, including red-teaming, explicit prompting, and implicit prompting. Additionally, we introduce a novel attack technique named SMILES-prompting, which uses the Simplified Molecular-Input Line-Entry System (SMILES) to reference chemical substances. Our findings reveal that SMILES-prompting can effectively bypass current safety mechanisms. These findings highlight the urgent need for enhanced domain-specific safeguards in LLMs to prevent misuse and improve their potential for positive social impact.
BibTeX
@misc{wong2024smiles,
  author = {Aidan Wong and He Cao and Zijing Liu and Yu Li},
  title = {{SMILES-Prompting: A Novel Approach to LLM Jailbreak Attacks in Chemical Synthesis}},
  year = {2024},
  eprint = {2410.15641},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.15641},
}
Figure