Omni-Mol: Exploring Universal Convergent Space for Omni-Molecular Tasks

Chengxin Hu, Hao Li, Yihe Yuan, Zezheng Song, Haixin Wang

TL;DR

Omni-Mol is a unified, instruction-tuned multimodal language model designed to handle 15 diverse chemical tasks simultaneously—from reaction prediction to molecular captioning. By introducing an adaptive gradient stabilization technique, an "anchor-and-reconcile" Mixture-of-Experts (MoE) architecture, and active-learning data filtering, it overcomes multi-task "conflict collapse" to beat much larger specialized models while using a compact 1.7B-parameter base.

What Problem Does This Paper Address?

Building a unified "AI chemist" requires a model to excel across diverse domains: predicting reaction outcomes, estimating quantum properties, describing molecular functions, and outputting lab synthesis protocols. However, simply mixing these datasets into a large language model (LLM) usually leads to conflict collapse:

  1. Representation and Optimization Conflicts: Different molecular tasks stress conflicting structural semantics, and softmax operations across tasks with vastly different entropies cause gradient norms to explode, destabilizing training in half-precision (bfloat16).
  2. Data Interference: Uncurated multi-task data mixtures can interfere destructively, making training computationally expensive and inefficient.
  3. Scalability Breakdown: Prior unified models (like InstructMol) actually see performance degrade as the number of tasks scales past a handful.

Omni-Mol tackles this core question: Can we build a generalist chemistry model that stably converges toward a universal molecular representation space?

Methods

Omni-Mol pairs LLaMA 3.2-1B with a MoleculeSTM 2D molecular graph encoder and a single linear projector, framing all tasks as multimodal instruction tuning:

  • Unified Encoding: The first molecule in a sequence is converted into graph tokens inserted into the prompt; additional molecules and targets are tokenized via string-based SELFIES.
  • Active Learning-Based Data Selection: Rather than training on all data equally, an iterative screening algorithm evaluates sample difficulty, down-weighting redundant data and reducing the training dataset to 40% of its original size with zero loss in performance.
  • Adaptive Gradient Stabilization: To prevent gradient explosion caused by softmax translation invariance across disparate tasks, the authors introduce a learnable, rank-normalized scaling factor ($\gamma_\theta = \frac{\alpha_\theta}{\|r\|_p} + \beta_\theta$) into LoRA updates.
  • Anchor-and-Reconcile MoE: By upcycling the upper dense feed-forward layers into a sparse MoE, Omni-Mol routes specialized task tokens among $\mathcal{N}$ "reconcile" experts while routing all tokens through an "anchor" expert that retains shared fundamental chemistry knowledge.

Datasets

The authors assembled an instruction-following omni-molecular benchmark containing 1.8 million samples across 15 tasks organized into four main categories:

  • Reaction Tasks: Forward reaction prediction, retrosynthesis, reagent prediction, catalyst prediction, and solvent prediction (derived from USPTO, USPTO_500_MT, and USPTO_500K).
  • Regression Tasks: Quantum mechanics property prediction (HOMO/LUMO from QM9), molecular weight, LogP, topological polar surface area (TPSA), molecular complexity (PubChem), SCF energy (PubChemQC), and reaction yield regression (RNX Yields).
  • Description Tasks: Molecular captioning (ChEBI-20) and Description Q&A (PubChem / 3D-MOIT).
  • Action Tasks: Experimental procedure synthesis protocol generation (USPTO-Applications and ORD).

Data was rigorously deduplicated across tasks to prevent cross-benchmark data leakage.

Key Insights and Ideas

  • Convergence to a Platonic Representation: As Omni-Mol trains on an increasing number of tasks (from 1 up to 8+), its learned representations across tasks become increasingly similar (measured via mutual $k$-NN). This directly supports the multitask scaling hypothesis, whereas previous baselines like InstructMol diverged.
  • Chemically Specialized Experts: Inspecting router activations reveals that specific functional groups naturally cluster into specific experts (e.g., ethers and amides route heavily to Expert 1, whereas imines and acyl chlorides route to Expert 2).
  • 40% Data is Plenty: Active data pruning eliminated redundant patent reactions while preserving difficult edge cases, yielding results that match or exceed full 100% data training.

Key Results

  • State-of-the-Art Performance: Despite having only 1.7B parameters, Omni-Mol outperforms much larger specialized 6.7B models and surpasses generalist baselines like PRESTO (3.2B / 6.7B):
  • Forward Reaction Prediction: 0.718 Exact Match vs. PRESTO’s 0.691.
  • Retrosynthesis: 0.559 Exact Match vs. PRESTO’s 0.531.
  • Reagent Prediction: 0.257 Exact Match vs. PRESTO’s 0.212.
  • Substantial gains were also observed in MolCap (0.456 BLEU-4) and Description Q&A (0.440 BLEU-4 vs. 0.261 for 3D-MoLM).
  • Inference Efficiency: Compared to PRESTO, Omni-Mol achieves a 65% reduction in latency, consumes 28% less GPU memory, generates tokens 1.78× faster, and cuts computation by 41% in GFLOPs.
  • Logarithmic Scaling Laws: Omni-Mol exhibits clear logarithmic scaling behaviors with respect to both model backbone size ($y = 0.02 \cdot \log(x) + 0.32$) and training dataset volume ($y = 0.07 \cdot \log(x) + 0.41$).

Limitations and Future Work

  • Continuous Regression Challenges: While Omni-Mol excels at generative text and SELFIES prediction, unified training slightly degraded performance on certain numerical regression tasks (such as molecular complexity and yield prediction) compared to task-specific training. The authors suggest that chain-of-thought reasoning or dedicated regression prediction heads will be necessary to resolve the gap between text generation and continuous scalar regression.
  • Graph Input Constraints: The current pipeline only converts the first molecule into a 2D graph token, processing the remainder purely as text SELFIES strings. Full multi-graph representation remains an area for future architectural expansion.

Summary updated

Abstract

Building generalist models has recently demonstrated remarkable capabilities in diverse scientific domains. Within the realm of molecular learning, several studies have explored unifying diverse tasks across diverse domains. However, negative conflicts and interference between molecules and knowledge from different domain may have a worse impact in threefold. First, conflicting molecular representations can lead to optimization difficulties for the models. Second, mixing and scaling up training data across diverse tasks is inherently challenging. Third, the computational cost of refined pretraining is prohibitively high. To address these limitations, this paper presents **Omni-Mol**, a scalable and unified LLM-based framework for direct instruction tuning. Omni-Mol builds on three key components to tackles conflicts: (1) a unified encoding mechanism for any task input; (2) an active-learning-driven data selection strategy that significantly reduces dataset size; (3) a novel design of the adaptive gradient stabilization module and anchor-and-reconcile MoE framework that ensures stable convergence. Experimentally, Omni-Mol achieves state-of-the-art performance across most of the 15 molecular tasks, demonstrates the presence of scaling laws in the molecular domain, and is supported by extensive ablation studies and analyses validating the effectiveness of its design. The dataset, code and weights of the AI-driven chemistry generalist are open-sourced.
BibTeX
@inproceedings{hu2025omni,
  author = {Chengxin Hu and Hao Li and Yihe Yuan and Zezheng Song and Haixin Wang},
  title = {{Omni-Mol: Exploring Universal Convergent Space for Omni-Molecular Tasks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=dw9H08UxJb},
}
Figure