Wasserstein Adversarial Autoencoders for Knowledge Graph Embedding based Drug-Drug Interaction Prediction

Yuanfei Dai, Chenhao Guo, Wenzhong Guo, Carsten EickhoffarXiv 2020 · arXiv · OpenReview

Abstract

Interaction between pharmacological agents can trigger unexpected adverse events. Capturing richer and more comprehensive information about drug-drug interactions (DDI) is one of the key tasks in public health and drug development. Recently, several knowledge graph embedding approaches have received increasing attention in the DDI domain due to their capability of projecting drugs and interactions into a low-dimensional feature space for predicting links and classifying triplets. However, existing methods only apply a uniformly random mode to construct negative samples. As a consequence, these samples are often too simplistic to train an effective model. In this paper, we propose a new knowledge graph embedding framework by introducing adversarial autoencoders (AAE) based on Wasserstein distances and Gumbel-Softmax relaxation for drug-drug interactions tasks. In our framework, the autoencoder is employed to generate high-quality negative samples and the hidden vector of the autoencoder is regarded as a plausible drug candidate. Afterwards, the discriminator learns the embeddings of drugs and interactions based on both positive and negative triplets. Meanwhile, in order to solve vanishing gradient problems on the discrete representation—an inherent flaw in traditional generative models—we utilize the Gumbel-Softmax relaxation and the Wasserstein distance to train the embedding model steadily. We empirically evaluate our method on two tasks, link prediction and DDI classification. The experimental results show that our framework can attain significant improvements and noticeably outperform competitive baselines.

Introduction: The Challenge of Negative Sampling in DDI

Predicting how two drugs might interact is a massive challenge in public health. Knowledge Graph Embedding (KGE) models have become a popular solution, representing drugs and their interactions as "triplets" (Head Drug, Interaction Type, Tail Drug) in a low-dimensional space.

Examples of drug-drug interaction triplets in a knowledge graph showing various effects like increased neuroexcitatory activities or decreased metabolism.

However, these models face a major hurdle: Negative Sampling. To learn what a real interaction looks like, the model needs to see fake ones. Most existing methods just pick a random drug from the database to create a fake triplet. These "easy fakes" are often so nonsensical that the model learns very little from them. Furthermore, using Generative Adversarial Networks (GANs) to create "smarter fakes" is notoriously difficult because drug IDs are discrete data points, which breaks the backpropagation used to train standard neural networks.

Methods: The Adversarial Autoencoder (AAE) Framework

The proposed framework consists of an Adversarial Autoencoder (AAE) architecture with three main components:

  1. The Encoder (Generator): It takes a partial real triplet (e.g., "Drug A, increases the effect of, ?") and generates a "plausible" drug to fill the gap.
  2. The Decoder: It ensures the generated drug stays semantically "close" to the original drugs in the feature space, preventing the generator from outputting random noise.
  3. The Discriminator (The KGE Model): This is the actual model being trained to predict DDIs. It is fed both real triplets and the "smart fakes" from the generator.
The proposed AAE framework architecture, featuring an encoder/generator, a decoder, and a discriminator utilizing Gumbel-Softmax relaxation.

To make this work, the authors employ two mathematical innovations:

  • Gumbel-Softmax Relaxation: This allows the model to "choose" a discrete drug ID in a way that is still differentiable, solving the backpropagation problem.
  • Wasserstein Distance: Instead of standard GAN loss, Wasserstein distance (Earth-Mover distance) is used to stabilize training and prevent "vanishing gradients," where the discriminator becomes too good for the generator to learn.

Experimental Results and Insights

The model was evaluated on two widely used public benchmarks: DeepDDI (1,710 drugs, 86 interaction types) and Decagon (637 drugs, 200 side effect types).

Key Findings

  • Superior Performance: The framework consistently outperformed competitive baselines like TransE, DistMult, and ComplEx. On the DeepDDI dataset, the model achieved a Mean Rank (MR) of 11.98, a significant improvement over the baseline ComplEx (15.84).
  • DDI Classification: The model showed a noticeable boost in PR-AUC (Precision-Recall Area Under Curve), particularly on the complex Decagon dataset, improving performance by roughly 1-2 percentage points.
  • Structural Learning: Visualization using UMAP showed that the model naturally clusters drugs by medical function (e.g., anti-fungals or corticosteroids) based solely on graph structure, successfully following the "translation invariance" principle.

Conclusion and Future Work

By using an adversary to generate deceptive negative samples, the model is forced to learn the nuanced features of drug interactions rather than just memorizing the graph. While the current model relies entirely on graph structure, future work could incorporate molecular features like chemical formulas or molecular weight to further boost accuracy. Additionally, scaling this framework to larger, noisier real-world clinical data remains a key challenge for the field.

Summary updated

Abstract

Interaction between pharmacological agents can trigger unexpected adverse events. Capturing richer and more comprehensive information about drug-drug interactions (DDI) is one of the key tasks in public health and drug development. Recently, several knowledge graph embedding approaches have received increasing attention in the DDI domain due to their capability of projecting drugs and interactions into a low-dimensional feature space for predicting links and classifying triplets. However, existing methods only apply a uniformly random mode to construct negative samples. As a consequence, these samples are often too simplistic to train an effective model. In this paper, we propose a new knowledge graph embedding framework by introducing adversarial autoencoders (AAE) based on Wasserstein distances and Gumbel-Softmax relaxation for drug-drug interactions tasks. In our framework, the autoencoder is employed to generate high-quality negative samples and the hidden vector of the autoencoder is regarded as a plausible drug candidate. Afterwards, the discriminator learns the embeddings of drugs and interactions based on both positive and negative triplets. Meanwhile, in order to solve vanishing gradient problems on the discrete representation--an inherent flaw in traditional generative models--we utilize the Gumbel-Softmax relaxation and the Wasserstein distance to train the embedding model steadily. We empirically evaluate our method on two tasks, link prediction and DDI classification. The experimental results show that our framework can attain significant improvements and noticeably outperform competitive baselines.
BibTeX
@misc{dai2020wasserstein,
  author = {Yuanfei Dai and Chenhao Guo and Wenzhong Guo and Carsten Eickhoff},
  title = {{Wasserstein Adversarial Autoencoders for Knowledge Graph Embedding based Drug-Drug Interaction Prediction}},
  year = {2020},
  eprint = {2004.07341},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2004.07341},
}
Figure