TL;DR
This paper introduces novel computational models that go beyond simply predicting if a compound binds to a protein, by also predicting whether it activates or inhibits that protein. By developing and comparing three different machine learning architectures, they demonstrate that cascaded models, particularly "Arch3," significantly improve the accuracy of predicting these functional effects, offering crucial insights for drug discovery and understanding complex biological mechanisms.
What Problem Does This Paper Address?
In drug discovery, identifying which protein a compound interacts with (target deconvolution) is a critical first step. However, knowing if a compound binds is often not enough. The functional effect of that binding – whether it activates or inhibits the protein – is crucial for understanding its mechanism of action and predicting its biological outcome. Existing in silico (computational) target prediction methods often fall short here, primarily focusing on predicting binding affinity rather than the specific functional consequence. This gap means researchers might miss crucial details about a compound's behavior, potentially leading to misinterpretations of phenotypic screening data or overlooking important off-target effects. For example, a compound might be predicted to bind a target, but without knowing if it activates or inhibits it, its role in a cellular pathway remains ambiguous.
Methods
The researchers developed and compared three distinct computational architectures for predicting functional effects:
- Architecture 1 (Arch1): A single, "global" Random Forest (RF) model trained on all available activating and inhibiting compound data across all targets. This serves as a baseline, representing a straightforward approach to functional prediction.
- Architecture 2 (Arch2): A two-stage cascaded model. Stage 1 uses an RF to predict if a compound binds to a target. If predicted to bind, Stage 2 uses another RF to predict whether the compound is activating or inhibiting.
- Architecture 3 (Arch3): Also a two-stage cascaded model, but with a key difference in Stage 2. Similar to Arch2, Stage 1 predicts binding. However, Stage 2 employs two independent RF models: one trained specifically on activating compounds and an inactive background set, and another trained on inhibiting compounds and the same inactive background set. The outputs of these two models are then compared to determine the functional effect. This architecture was hypothesized to better handle the imbalance between activating and inhibiting data.
All models utilized Random Forests, a standard machine learning algorithm. The key novelty lies in the cascaded architectures (Arch2 and Arch3) and the specific design of Arch3's Stage 2, which incorporates inactive compounds as a reference.
Datasets
The study utilized a substantial dataset compiled from AstraZeneca's internal bioactivity data, primarily from the Chemistry Connect repository.
- Training Data:
- Binding Data: 817,239 active data points with unknown functional effect (binding only).
- Inactive Compounds: 20,761,260 compounds.
- Functional Data: 226,045 activating and 1,032,439 inhibiting data points from functional screens.
- These datasets covered 332 protein targets.
- Prospective Validation Data: A temporal set of 153,467 in-house data points, collected 4 months after the initial training data extraction, spanning 306 targets. This dataset was used to assess how well the models generalize to new data over time.
- Inactive Compounds: A large set of inactive compounds was also sourced from PubChem, mapped to bioactivity assay IDs, and supplemented with AstraZeneca's high-throughput screen data.
Key Insights and Ideas
- Functional Effects Matter: The chemical space analysis revealed that binding and inhibiting compounds are more chemically similar than binding and activating compounds. This suggests distinct underlying mechanisms and provides a strong rationale for developing models that differentiate these functional outcomes.
- Cascading is Superior: The cascaded architectures (Arch2 and Arch3) consistently outperformed the single-model approach (Arch1) in both cross-validation and prospective validation. This highlights the benefit of breaking down the prediction problem into sequential steps (binding first, then functional effect).
- Inactive Background is Key: Architecture 3, which explicitly used inactive compounds in its Stage 2 functional prediction models, demonstrated superior performance. This suggests that having a clear reference point for "non-activity" is crucial for accurately distinguishing between activation and inhibition.
- Imbalance is a Challenge: The study acknowledges the inherent imbalance in functional data (often far more inhibiting than activating data). Arch3's design appears to handle this better, leading to improved performance, particularly for the minority activating class.
- Applicability Domain is Important: The distance-based applicability domain analysis showed that Arch3 has a superior ability to extrapolate into novel chemical spaces, meaning its predictions are more reliable for compounds that are structurally different from the training data.
Key Results
- Cross-Validation: Arch3 achieved the highest target-averaged precision (92.0 ± 9.1%) and recall (82.9 ± 11.6%) during cross-validation. Arch1 performed the lowest with 84.5 ± 12.1% precision and 68.7 ± 17.5% recall.
- Prospective Validation: While the performance gap narrowed, Arch3 still showed strong results with class-averaged precision of 70.8 ± 3.5% and recall of 53.1 ± 3.6%. Arch1 lagged behind with 59.5 ± 3.2% precision and 48.1 ± 1.3% recall.
- Applicability Domain: Arch3 demonstrated the highest Area Under the Curve (AUC) of 0.30 for its distance-based applicability domain, indicating better generalization to new chemical spaces compared to Arch1 (AUC = 0.22) and Arch2 (AUC = 0.29).
- Case Study (CHRM1): The models were applied to a real-world scenario (Sertindole and CHRM1 receptor) where functional prediction provided crucial insights that were missed by previous analyses, highlighting the practical utility of the developed methods.
Limitations and Future Work
- Data Imbalance: While Arch3 mitigates the issue, the inherent imbalance between activating and inhibiting data remains a challenge, particularly for minority classes and certain target classes like ion channels and transporters.
- Extrapolation Limits: All models, even Arch3, struggle to extrapolate reliably into highly dissimilar chemical spaces (distances beyond 0.6 in the AD analysis).
- Minimum Data Requirements: The study suggests that datasets with fewer than 25 compounds for a given target may lead to poor performance, indicating a need for sufficient data to train robust models.
- Future Work: The authors propose deploying these models in-house to aid future phenotypic screening analyses. They also suggest that incorporating functional effects could significantly improve similar analyses in the future, helping to identify unanticipated functional changes. The case study on Sertindole and CHRM1 receptor exemplifies this potential.
Summary updated