Unified Foundation Models for Molecular Discovery

42 concepts44 papers

In machine learning for the chemical and biological sciences, a persistent bottleneck has been task-specific model fragmentation. Historically, if a researcher wanted to predict a molecule's solubility, evaluate its toxicity, generate a new molecular scaffold, or design an antibody loop, they had to train separate, highly specialized models. Each task required its own architecture, its own training pipeline, and its own isolated dataset.

This fragmentation stems from a fundamental design choice: traditional models map a specific input format (such as a 2D SMILES string) directly to a single target property. By bypassing the underlying physical and structural realities that govern all chemical behavior, these models become rigid specialists, unable to share knowledge across tasks or generalize to new scenarios.

To overcome this rigidity, a powerful paradigm shift is underway. Instead of building disconnected specialists, researchers are developing unified foundation models. By learning fundamental physical states, continuous spatial representations, and multi-task chemical languages, these generalist architectures can handle the full spectrum of molecular discovery within a single framework.


Learning the Fundamental Quantum State

The most direct way to eliminate property-specific fragmentation is to change what the model learns. Every macroscopic chemical property—whether it is binding affinity, dipole moment, or chemical reactivity—is an expression of a molecule's underlying electronic structure.

Instead of training separate models to predict individual scalar properties, we can train a single model to predict the fundamental quantum mechanical state of the molecule.

Traditional ML:  [Molecule] ───► Model A ───► Solubility
                 [Molecule] ───► Model B ───► Toxicity

Unified ML:      [Molecule] ───► Unified Model ───► Wavefunction ───► All Properties

As demonstrated by Schütt et al. et al. in Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions, a deep neural network can be designed to learn the molecular wavefunction and the underlying Hamiltonian and overlap matrices. Because the wavefunction contains the complete physical description of the system, any electronic property can be computed directly from it. This approach achieves the accuracy of ab initio quantum chemistry at the speed of a classical force field, providing a universal representation that generalizes across diverse chemical phenomena.

To make this prediction computationally viable, the architecture must respect the laws of physics. Unke et al. et al. expanded on this in SE(3)-equivariant prediction of molecular wavefunctions and electronic densities by hard-coding rotational and translational symmetries ($\text{SE}(3)$-equivariance) directly into the network. By ensuring that the model's internal representations rotate and shift exactly as the physical molecule does in real space, the authors achieved a 1,000x speedup over traditional quantum mechanics methods while drastically reducing prediction errors.


Unifying Modalities via Continuous Fields and Block Graphs

Model fragmentation also occurs across different molecular classes. In traditional workflows, small molecules are treated as discrete graphs, while proteins are treated as long sequences of amino acid residues. This "one modality, one model" split prevents architectures from learning the universal principles of atomic interactions that apply to all matter.

To bridge this gap, recent work unifies how different molecular types are represented. Rather than forcing molecules into rigid, human-defined discrete categories, architectures can treat them as continuous distributions in space.

Discrete Representation:   [Small Molecule Graph]  vs  [Protein PDB Backbone]  --> (Two separate models)
Continuous Representation: [Atomic Density Field]  ==  [Atomic Density Field]  --> (One unified model)

In Unified all-atom molecule generation with neural fields, Kirchmeyer et al. et al. introduced a framework that uses neural fields to represent molecules as continuous atomic densities. Because every biological entity—whether a small molecule drug, a peptide, or an antibody loop—is ultimately just a collection of atoms in 3D space, this continuous representation allows a single AI model to design across all of these modalities simultaneously.

A parallel approach simplifies structural biology tasks like inverse folding (designing an amino acid sequence that fits a specific 3D shape). Gao et al. et al. developed UniIF: Unified Molecule Inverse Folding, which standardizes structural data from proteins, RNA, and small materials into a shared format called "block graphs." By using a single geometric attention network across these standardized graphs, the model learns universal structural principles, eliminating the need for domain-specific geometric featurization strategies.


Integrating 2D Topologies and 3D Geometries

Even when focusing on a single molecule, models are frequently fragmented by data format. Some models excel at processing 2D topological graphs (useful for rapid property screening), while others are built for 3D geometric coordinates (essential for understanding physical interactions).

2D Graph (Connectivity) ──┐
                          ├──► [Shared Transformer Backbone] ──► Multi-Task Predictions
3D Geometry (Coordinates) ──┘

Luo et al. et al. resolved this dichotomy with Transformer-M, detailed in One Transformer Can Understand Both 2D & 3D Molecular Data. The architecture utilizes a shared Transformer backbone equipped with specialized channels to ingest both 2D and 3D data. When 3D coordinates are missing, the model activates its 2D channels; when geometric data is available, it leverages spatial distances to enrich its representations. This joint training allows the model to develop a deeper, format-agnostic understanding of molecular properties.

For generative tasks, Kuznetsov developed nach0-pc: Multi-task Language Model with Molecular Point Cloud Encoder. By pairing a multi-task language model with an order-invariant 3D point cloud encoder, this single architecture can execute 3D molecule generation, scaffold decoration, and linker design—tasks that previously required a suite of specialized, non-interoperable diffusion models.


Chemical Language Models as Generalist Assistants

The ultimate expression of this unification strategy is the chemical foundation model. By treating chemical structures, properties, and textual descriptions as tokens in a shared language, these models act as a centralized "chemical brain" capable of simultaneous reasoning, generation, and prediction.

In BARTSmiles: Generative Masked Language Models for Molecular Representations, Chilingaryan et al. et al. demonstrated that a large-scale generative masked language model trained on billions of molecules breaks the traditional barrier between discriminative tasks (like property prediction) and generative tasks (like molecular design). During pre-training, the model naturally develops internal features that identify key chemical substructures, allowing it to excel at both predicting toxicity and generating novel compounds.

                      ┌──► Property Prediction (Solubility, Toxicity)
[Chemical Foundation  ├──► Structure Generation (Scaffold Hopping)
       Model]         └──► Workflow Automation (Retrosynthesis Planning)

This concept scales up significantly with models like ChemFM, introduced by Cai et al. et al. in ChemFM as a scaling law guided foundation model pre-trained on informative chemicals. Guided by scaling laws to maximize data efficiency, this 3-billion-parameter model unifies property prediction, molecular generation, and chemical reaction forecasting within a single architecture.

To orchestrate these capabilities in practical research, frameworks like MMAI Gym for Science, proposed by Kuznetsov et al. et al. in MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery, train foundation models to perform holistic reasoning across the entire drug development pipeline. Instead of relying on disconnected specialist tools, a single model can evaluate a compound's safety, optimize its structure, and plan its chemical synthesis in a cohesive, autonomous loop.

By replacing fragmented, single-purpose architectures with unified representations, the molecular sciences are moving toward a future where a single generalist model can understand, manipulate, and design the full complexity of the physical world.

Key Foundational Work

Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions (2019) — contributed 1 concept elements to this direction.
Learning to Design RNA (2019) — contributed 1 concept elements to this direction.
SE(3)-equivariant prediction of molecular wavefunctions and electronic densities (2021) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_task_specific_model_fragmentation_task_s_65716021,
  title = {task-specific model fragmentation (task-specific model rigidity)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/task-specific-model-fragmentation-task-specific-model-rigidity/rdr_6571602196a84fe595d16bb3d3cef5e1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_task_specific_model_fragmentation_task_s_65716021_v1,
  title = {task-specific model fragmentation (task-specific model rigidity)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/task-specific-model-fragmentation-task-specific-model-rigidity/rdr_6571602196a84fe595d16bb3d3cef5e1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake