Abstract
Predicting drug-target interactions (DTIs) is a critical task in drug discovery, traditionally requiring significant time and financial investment. While Knowledge Graph Embedding (KGE) models have accelerated this process, they often suffer from poor calibration, where predicted scores do not accurately reflect the true probability of an interaction. This paper introduces a Causal Intervention (CI) framework that assesses prediction confidence by measuring the robustness of model representations. By systematically perturbing embeddings and evaluating the consistency of the model's output, the proposed method provides a more reliable measure of certainty. Evaluations on the Hetionet and BioKG datasets show that CI effectively reduces calibration error and enhances the reliability of top-tier predictions.
Introduction
In drug discovery, identifying Drug-Target Interactions (DTIs) is analogous to finding the correct key for a biological lock. Knowledge Graph Embedding (KGE) models represent drugs and proteins as high-dimensional vectors to predict potential links. However, these models frequently exhibit overconfidence or underconfidence, a state known as being "poorly calibrated." Standard confidence measures, such as simple softmax scores, are often uninterpretable and unstable. This research addresses the "confidence problem" by developing a "trust meter" based on causal inference principles, ensuring that high model scores correspond to high actual accuracy.
The Causal Intervention Framework
The authors propose a multi-stage framework for confidence measurement centered on causal intervention, moving beyond simple output scores to evaluate the underlying logic of the model's predictions.
- Embedding & Prediction: Standard KGE architectures (e.g., TransE, RotatE) learn latent representations of drugs and targets.
- Causal Intervention: Instead of relying solely on the raw output, the system intervenes on the drug's embedding vector. By perturbing individual dimensions, "pseudo-entities" are created.
- Consistency Scoring: The model re-evaluates interactions using these perturbed vectors. Robust predictions (high confidence) should maintain stable rankings despite these small changes. High fluctuation indicates the model is reacting to "noise" rather than a solid biological signal.
- Probabilistic Calibration: Consistency scores are mapped to actual probability values using techniques like Platt Scaling and Isotonic Regression.
Experimental Evaluation
The method was evaluated using two major biomedical knowledge graphs: Hetionet, an integrative network of diseases and compounds, and BioKG, a benchmark dataset for life sciences. The researchers tested the framework across six KGE architectures (TransE, Complex, DisMult, RotatE, TransH, and CrossE) to ensure generalizability.
Key Results and Insights
The core finding is that robustness equals reliability. A genuine biological relationship is reflected in the core semantic structure of the embedding; if a minor tweak changes the model's prediction, the original result was likely unreliable.
- Reduced Calibration Error: The Causal Intervention (CI) method consistently achieved lower Expected Calibration Error (ECE) compared to baselines like SigmoidMax or TopKSoftmax across both datasets.
- Improved Top-Tier Accuracy: On the Hetionet dataset with the CrossE model, the CI method nearly doubled the accuracy of the top 10% most confident predictions (T10ACC) compared to standard baselines.
- Effective Calibration: As shown in the reliability diagrams, applying Isotonic Regression to causal scores results in predictions that closely follow the ideal diagonal, where predicted probability matches actual accuracy.
Conclusion and Future Work
The Causal Intervention method provides a practical tool for researchers to prioritize drug candidates for clinical trials by offering a more stable and accurate confidence distribution. However, challenges remain regarding dimensionality sensitivity—the method currently performs better on lower-dimensional embeddings—and computational overhead due to the need for multiple re-scorings. Future research will focus on automating the selection of intervention dimensions and adapting the framework for multi-modal biological data.
Summary updated