The rapid advancement of machine learning in chemistry and biology has unlocked unprecedented capabilities for drug discovery, protein engineering, and materials science. However, the mathematical foundations that enable these models to discover life-saving therapeutics also introduce a profound dual-use risk. The exact same mechanisms used to optimize a protein's binding affinity to a disease target can be inverted or repurposed to engineer lethal toxins, hazardous chemicals, or novel viral agents.
As biochemical AI tools become more powerful and accessible, the machine learning community is shifting from purely performance-driven development toward the design of robust biosecurity guardrails, red-teaming frameworks, and safety-constrained generation algorithms.
The Mechanics of the Dual-Use Dilemma
To understand why biomolecular models are uniquely susceptible to dual-use exploitation, we must look at how they represent and navigate chemical space. High-fidelity representation models—such as the multi-view framework proposed by Luo et al. in Learning Multi-view Molecular Representations with Structured and Unstructured Knowledge or the geometric transformer developed by Chen et al. in GeoMFormer: A General Architecture for Geometric Molecular Representation Learning—map discrete atomic arrangements into continuous, searchable latent spaces. These models are designed to capture complete structural and functional properties, making them dual-use by nature: a model that understands how a protein folds to cure a disease also understands how it folds to cause harm.
This risk is amplified by two compounding factors:
- Lowered Technical Barriers: Large Language Models (LLMs) trained on massive biomolecular instruction datasets, such as the dataset curated by Fang et al. in Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models, bridge the gap between general-purpose AI and specialized expertise. Multi-agent frameworks like the one proposed by Lv et al. in ProteinEngine: Empower LLM with Domain Knowledge for Protein Engineering allow users to orchestrate complex biological tools through natural language, lowering the barrier to entry for malicious actors.
- Autonomous Execution: Agentic systems are evolving to act independently. As demonstrated by Wölflein et al. in LLM Agents Making Agent Tools, autonomous agents can now read scientific literature and synthesize their own software tools to execute complex workflows, raising the prospect of automated biochemical design operating entirely without human oversight.
Why Optimization Accelerates the Threat
The core engine of molecular design is optimization. Whether a system uses fragment-based retrieval (such as the frameworks designed by Lee et al. in Molecule Generation with Fragment Retrieval Augmentation and Drug Discovery with Dynamic Goal-aware Fragments), out-of-distribution diffusion (like the framework presented by Lee et al. in Exploring Chemical Space with Score-based Out-of-distribution Generation), or atom-by-atom generation (as explored by Pandey in Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation), the goal is to navigate chemical space to maximize a specific reward function.
To make this search efficient, researchers have focused on smoothing the optimization landscape. For instance, the architectures developed by Lee et al. in Latent Bayesian Optimization via Autoregressive Normalizing Flows and Advancing Bayesian Optimization via Learning Correlated Latent Space explicitly align latent space geometry with objective values to eliminate discrepancies and accelerate discovery.
[Standard Optimization Pipeline]
Latent Space Exploration ──> Reward Maximization ──> High-Affinity Candidates
│
▼
(Potential Toxic Compounds)
The danger is that these optimization algorithms are entirely indifferent to ethics. If a model lacks integrated safety constraints or toxicity filters, a smoothed, highly efficient search landscape will find highly potent, hazardous compounds just as rapidly as it would find a cure. Furthermore, zero-shot structural editing models, such as the one proposed by Chen et al. in Manipulating 3D Molecules in a Fixed-Dimensional E(3)-Equivariant Latent Space, allow users to alter molecular properties via simple vector math, making it trivial to bypass superficial, post-hoc safety filters.
Probing Vulnerabilities via Red-Teaming
To defend against these risks, researchers have begun systematically "red-teaming" frontier models to expose their biosecurity vulnerabilities.
In the domain of proteomics, Fan et al. introduced SafeProtein in SafeProtein: Red-Teaming Framework and Benchmark for Protein Foundation Models. By using multimodal prompts and advanced search strategies, the authors demonstrated that state-of-the-art protein foundation models like ESM3 can be jailbroken to reconstruct harmful toxins and viruses with success rates as high as 70%. This proved that existing safety measures are insufficient to prevent the generation of dangerous biological sequences.
On the chemical side, evaluating these risks requires benchmarking AI against human expertise. Mirza et al. established ChemBench in A framework for evaluating the chemical knowledge and reasoning abilities of large language models against the expertise of chemists, pitting LLMs against human chemists. Their findings highlighted a critical tension in biosecurity: while frontier models now outperform average human experts in chemical reasoning, standard safety guardrails designed to block malicious queries often inadvertently block legitimate scientific research into toxicology and risk assessment.
Designing Provable Safeguards
How do we build models that can safely discover drugs without risking the synthesis of bioweapons? The consensus is moving away from superficial text-based filters and toward provable, physics-grounded constraints embedded directly into the generative process.
For sequence-based generation, frameworks like the one proposed by Yi et al. in Graph Denoising Diffusion for Inverse Protein Folding or the reinforcement learning approach developed by Cao et al. in GLID$^2$E: A Gradient-Free Lightweight Fine-tune Approach for Discrete Biological Sequence Design demonstrate how reward shaping and evolutionary priors can guide generation. To mitigate dual-use risks, these reward functions can be modified to penalize structural similarity to known pathogens or toxins.
[Safety-Constrained Pipeline]
Latent Space Exploration ──> Neuro-Symbolic Filter ──> Projected Safe Subspace ──> Verified Therapeutics
▲
│
(Safety Protocols & Physics Laws)
The most robust approach involves mathematically restricting the model's output space. In How can we ground Frontiers Models for Biological Research to Domain Knowledge, First Principles, and Constraints?, Fioretto proposes Projected Generative Diffusion Models (PGDM). Instead of filtering out toxic compounds after they are generated, PGDM uses neuro-symbolic integration to project the model's intermediate states during the diffusion process onto a safe subspace that strictly satisfies predefined safety protocols and physical laws.
By ensuring that the generative mechanics themselves cannot mathematically violate safety constraints, researchers are paving the way for AI-driven discovery tools that can be safely trusted in automated experimental laboratories.
Key Foundational Work
Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generation (2021) — contributed 1 concept elements to this direction.
Periodic Graph Transformers for Crystal Material Property Prediction (2022) — contributed 1 concept elements to this direction.
Large-scale chemical language representations capture molecular structure and properties (2022) — contributed 1 concept elements to this direction.