Fate-tox: fragment attention transformer for E(3)-equivariant multi-organ toxicity prediction

Sumin Ha, Dongmin Bang, Sun KimJ. Cheminformatics 2025 · DOI

TL;DR

This paper introduces FATE-Tox, a novel deep learning framework that predicts multi-organ toxicity by integrating 2D and 3D molecular representations with a fragment-attention transformer. It achieves state-of-the-art results on several benchmark datasets and offers interpretable insights into potential toxicophores, addressing a critical need for safer drug development.

What Problem Does This Paper Address?

Drug development is plagued by toxicity, often leading to late-stage failures. While many computational models predict toxicity for a single organ, this approach is insufficient because mitigating toxicity in one organ can inadvertently lead to adverse effects in another (e.g., rofecoxib). Predicting toxicity across multiple organs simultaneously is crucial but challenging due to:

  1. Variability of toxic substructures: Different molecular fragments can cause toxicity in different organs through distinct mechanisms.
  2. Limited molecular representations: Existing models often struggle to capture the full picture of molecular interactions, which can involve both 2D structural information and 3D spatial arrangements.
  3. Lack of explainability: Understanding why a molecule is predicted to be toxic, especially in terms of specific substructures (toxicophores), is vital for drug design but often missing in current models.

Methods

FATE-Tox tackles these challenges with a multi-pronged approach:

  • Dual-Branch Architecture: The model employs two parallel branches:
  • Atom-level branch: Focuses on fine-grained interactions between individual atoms.
  • Fragment-level branch: Captures broader molecular structures by breaking down molecules into fragments.
  • Fragmentation Methods: To address the variability of toxic substructures, the fragment-level branch utilizes three distinct fragmentation strategies:
  • BRICS: Breaks molecules into smaller, chemically meaningful fragments.
  • Bemis-Murcko scaffolds: Identifies core ring structures and peripheral side chains.
  • RDKit Functional Groups: Isolates specific reactive chemical groups. This multi-view fragmentation provides a more comprehensive representation of molecular substructures.

  • Transformer and E(3)-Equivariant Graph Neural Networks (EGNN):

    • A Molecular Transformer module processes 2D graph features (atom and bond information) and incorporates adjacency and distance matrices to refine atom embeddings. This module is inherently E(3)-invariant.
  • An E(3)-equivariant graph convolutional layer (EGCL) then integrates 3D spatial coordinates. The E(3)-equivariance ensures that the model's predictions are invariant to rotations and translations of the molecule's 3D structure, crucial for physically meaningful predictions.
  • Multi-Task Learning (MTL) with PCGrad: The model is trained using a multi-task learning setup to predict multiple toxicity endpoints simultaneously. To mitigate conflicting gradients during training (a common issue in MTL), they employ PCGrad (Projecting Conflicting Gradients), which modifies gradients to ensure balanced optimization across tasks.
  • Explainability: The fragment attention transformer mechanism allows for visualization of attention coefficients, highlighting substructures that contribute most to toxicity predictions.

Datasets

The study evaluated FATE-Tox on a range of established toxicity datasets from:

  • MoleculeNet: BBBP (Blood-Brain Barrier Penetration), SIDER (Side Effects).
  • Therapeutics Data Commons (TDC): ClinTox (Clinical Toxicology), DILI (Drug-Induced Liver Injury), Skin Reaction, Carcinogens, HERG (hERG channel inhibition).

These datasets cover various organ-specific and systemic toxicity endpoints. The Moleculenet datasets were split using scaffold splitting, ensuring that the test set contained unseen Murko scaffolds to evaluate generalizability. TDC datasets used their provided scaffold-splitted subsets.

Key Insights and Ideas

  • Synergy of 2D, 3D, and Fragments: The most significant insight is that combining 2D topological information, 3D spatial arrangements, and multiple fragment-based representations is crucial for robust multi-organ toxicity prediction. No single representation is sufficient.
  • Fragment Importance Varies by Endpoint: Different fragmentation methods contribute differently to predicting various toxicity endpoints, underscoring the need for a flexible, multi-fragment approach. For example, BRICS was important for BBBP, while RDKit functional groups were key for reactivity-driven toxicities.
  • E(3)-Equivariance is Key for 3D: Incorporating E(3)-equivariant layers is essential for accurately leveraging 3D molecular information, especially for distinguishing between enantiomers with different biological activities (like thalidomide).
  • MTL Enhances Generalization and Embeddings: Multi-task learning not only improves prediction performance but also leads to richer, more discriminative embedding spaces that better capture shared knowledge across tasks.
  • Attention for Toxicophore Identification: The attention mechanism provides a powerful tool for identifying specific molecular substructures that are responsible for toxicity, offering a level of interpretability rarely seen in complex deep learning models.

Key Results

  • State-of-the-Art Performance: FATE-Tox achieved state-of-the-art results on seven benchmark toxicity datasets, outperforming various baseline models, including advanced graph neural networks and transformers.
  • Performance Gains: The framework showed significant improvements, with up to 3.01% gains over prior baseline methods. The multi-task learning approach further enhanced performance by up to 1.44% compared to its single-task counterpart.
  • Improved Embeddings: Visualizations showed that FATE-Tox's multi-view and multi-task embeddings led to better-separated clusters, indicating superior representation learning.
  • Enantiomer Discrimination: The model successfully distinguished between (R)- and (S)-thalidomide, a feat not achieved by 2D-only models, highlighting the importance of 3D information.
  • Interpretable Toxicophore Identification: Case studies demonstrated that FATE-Tox's attention visualizations aligned with known toxicophores and literature, providing credible explanations for predictions.

Limitations and Future Work

  • Adaptive Fragmentation: The authors suggest future work could explore adaptive fragmentation strategies that dynamically adjust based on the chemical context, potentially further improving model flexibility and performance.
  • Time-Dependent Toxicity: Extending the model to account for time-dependent toxicity patterns, such as bioaccumulation or metabolic pathways, is proposed to provide a more comprehensive understanding of long-term chemical effects.
  • Broader Applications: The vision is to develop a comprehensive, interpretable, and scalable system for toxicity prediction that bridges the gap between in-silico modeling and real-world applications.

Summary updated

BibTeX
@article{ha2025fate,
  author = {Sumin Ha and Dongmin Bang and Sun Kim},
  title = {{Fate-tox: fragment attention transformer for E(3)-equivariant multi-organ toxicity prediction}},
  journal = {J. Cheminformatics},
  year = {2025},
  doi = {10.1186/s13321-025-01012-5},
}
Figure