Research And Implementation Of Drug Target Interaction Confidence Measurement Method Based On Causal Intervention

Wenting Ye, Bowen Wang, Yang Xie, Debo Cheng, Zaiwen Feng

Abstract

Predicting drug-target interactions (DTIs) is a critical task in drug discovery, traditionally requiring significant time and financial investment. While Knowledge Graph Embedding (KGE) models have accelerated this process, they often suffer from poor calibration, where predicted scores do not accurately reflect the true probability of an interaction. This paper introduces a Causal Intervention (CI) framework that assesses prediction confidence by measuring the robustness of model representations. By systematically perturbing embeddings and evaluating the consistency of the model's output, the proposed method provides a more reliable measure of certainty. Evaluations on the Hetionet and BioKG datasets show that CI effectively reduces calibration error and enhances the reliability of top-tier predictions.

Introduction

In drug discovery, identifying Drug-Target Interactions (DTIs) is analogous to finding the correct key for a biological lock. Knowledge Graph Embedding (KGE) models represent drugs and proteins as high-dimensional vectors to predict potential links. However, these models frequently exhibit overconfidence or underconfidence, a state known as being "poorly calibrated." Standard confidence measures, such as simple softmax scores, are often uninterpretable and unstable. This research addresses the "confidence problem" by developing a "trust meter" based on causal inference principles, ensuring that high model scores correspond to high actual accuracy.

The Causal Intervention Framework

The authors propose a multi-stage framework for confidence measurement centered on causal intervention, moving beyond simple output scores to evaluate the underlying logic of the model's predictions.

  1. Embedding & Prediction: Standard KGE architectures (e.g., TransE, RotatE) learn latent representations of drugs and targets.
  2. Causal Intervention: Instead of relying solely on the raw output, the system intervenes on the drug's embedding vector. By perturbing individual dimensions, "pseudo-entities" are created.
  3. Consistency Scoring: The model re-evaluates interactions using these perturbed vectors. Robust predictions (high confidence) should maintain stable rankings despite these small changes. High fluctuation indicates the model is reacting to "noise" rather than a solid biological signal.
  4. Probabilistic Calibration: Consistency scores are mapped to actual probability values using techniques like Platt Scaling and Isotonic Regression.
The overall framework of the Causal Intervention method for DTI confidence measurement.

Experimental Evaluation

The method was evaluated using two major biomedical knowledge graphs: Hetionet, an integrative network of diseases and compounds, and BioKG, a benchmark dataset for life sciences. The researchers tested the framework across six KGE architectures (TransE, Complex, DisMult, RotatE, TransH, and CrossE) to ensure generalizability.

Key Results and Insights

The core finding is that robustness equals reliability. A genuine biological relationship is reflected in the core semantic structure of the embedding; if a minor tweak changes the model's prediction, the original result was likely unreliable.

  • Reduced Calibration Error: The Causal Intervention (CI) method consistently achieved lower Expected Calibration Error (ECE) compared to baselines like SigmoidMax or TopKSoftmax across both datasets.
  • Improved Top-Tier Accuracy: On the Hetionet dataset with the CrossE model, the CI method nearly doubled the accuracy of the top 10% most confident predictions (T10ACC) compared to standard baselines.
  • Effective Calibration: As shown in the reliability diagrams, applying Isotonic Regression to causal scores results in predictions that closely follow the ideal diagonal, where predicted probability matches actual accuracy.
Reliability diagrams comparing different calibration methods (Uncalibrated, Platt, Isotonic) across Hetionet and BioKG datasets.

Conclusion and Future Work

The Causal Intervention method provides a practical tool for researchers to prioritize drug candidates for clinical trials by offering a more stable and accurate confidence distribution. However, challenges remain regarding dimensionality sensitivity—the method currently performs better on lower-dimensional embeddings—and computational overhead due to the need for multiple re-scorings. Future research will focus on automating the selection of intervention dimensions and adapting the framework for multi-modal biological data.

Summary updated

BibTeX
@inproceedings{ye2023research,
  author = {Wenting Ye and Bowen Wang and Yang Xie and Debo Cheng and Zaiwen Feng},
  title = {{Research And Implementation Of Drug Target Interaction Confidence Measurement Method Based On Causal Intervention}},
  booktitle = {BIBM},
  year = {2023},
  url = {https://openreview.net/forum?id=4yy2zoONaE},
}
Figure