siRNADesign: A Graph Neural Network for siRNA Efficacy Prediction via Deep RNA Sequence Analysis

Rongzhuo Long, Ziyu Guo, Da Han, Xudong Yuan, Guangyong Chen, Pheng‐Ann Heng, Liang ZhangbioRxiv 2024 · DOI

TL;DR

This paper introduces siRNADesign, a novel Graph Neural Network (GNN) that significantly improves the prediction of siRNA efficacy by incorporating a richer set of RNA sequence features and interactions. To ensure robust evaluation, they also propose a new data-splitting methodology that mitigates data leakage, setting a new standard for future research in this field.

What Problem Does This Paper Address?

Small interfering RNAs (siRNAs) are powerful tools for gene silencing, but predicting which siRNA will be most effective is a major challenge. Existing computational methods often overlook crucial aspects of the siRNA silencing mechanism, such as the precise positioning of siRNA on its target mRNA, the probabilities of base-pairing, and the interaction with key proteins like AGO2. This incomplete understanding limits the accuracy and reliability of current siRNA design tools, leading to wasted experimental effort and reduced therapeutic potential.

Methods

The core of siRNADesign is a Graph Neural Network (GNN) that models the complex interactions between siRNAs and their target mRNAs. The novelty lies in the comprehensive feature extraction and the GNN architecture:

  • Rich Feature Engineering: The model leverages both "non-empirical" features (like one-hot sequence encoding, positional embeddings, and RNA-protein interaction probabilities) and "empirical rule-based" features (such as thermodynamic stability, nucleotide frequencies, GC content, and specific siRNA rule codes). This broad set of features aims to capture the multifaceted nature of gene silencing.
  • Graph Representation: siRNADesign constructs a heterogeneous graph where nodes represent siRNA, mRNA, and their interactions. This graph structure allows the GNN to learn complex relationships and dependencies between these components.
  • Novel Data Splitting: To address the critical issue of data leakage (where test data inadvertently influences training), the authors propose a new evaluation strategy. Instead of traditional k-fold cross-validation, they randomly split data into training, validation, and testing sets (70:15:15) and crucially, they perform this split based on either siRNA or mRNA identity to prevent related sequences from appearing in both training and testing sets. This approach is repeated with different random seeds to ensure robustness.

Datasets

The study utilized several datasets for training and evaluation:

  • Dataset_HUVK: A large dataset comprising 2816 siRNA-mRNA pairs, collected from previous studies.
  • Simone Dataset: An external dataset with 322 siRNAs, used for validation.
  • In-house Dataset: A custom dataset generated from in vitro wet lab experiments, focusing on the SERPINC1 gene, which included 102 siRNAs with consistent silencing efficiencies.

The paper highlights their newly proposed data-splitting methodology as a benchmark for future research, differing from the standard 10-fold cross-validation used previously.

Key Insights and Ideas

  • The Power of Comprehensive Features: The study strongly suggests that incorporating a wide array of features, including those previously overlooked (like RNA-AGO2 interactions and positional information), is crucial for accurate siRNA efficacy prediction.
  • Graph Neural Networks Excel at Complex Interactions: GNNs are well-suited to model the intricate relationships within biological systems, making them a powerful tool for understanding gene silencing mechanisms.
  • Data Leakage is a Serious Problem: The authors emphasize that traditional cross-validation methods can lead to inflated performance metrics due to data leakage, underscoring the importance of their proposed robust splitting strategy.
  • Empirical Rules Still Matter: While deep learning is powerful, integrating well-established empirical rules (like nucleotide frequencies) still provides significant performance gains.

Key Results

  • State-of-the-Art Performance: siRNADesign achieved state-of-the-art performance on internal datasets, outperforming existing methods like GNN4siRNA, DSIR, s-Biopredsi, i-Score, and a CNN-based model. For example, on the siRNA-split Dataset_HUVK, it reached an average PCC of 0.770, SPCC of 0.771, and AUC of 0.874.
  • Superior Generalization: The model demonstrated strong performance on external and in-house datasets, including in vitro wet lab experiments, indicating its robustness and generalizability.
  • Validation of New Splitting Method: The proposed data-splitting methodology showed consistent results and provided a more realistic evaluation of model performance.

Limitations and Future Work

  • Future Directions: The authors plan to explore chemical modifications of siRNAs and integrate transformer models and other advanced deep-learning architectures to further enhance predictive accuracy.
  • Ablation Study Insights: While not strictly a limitation, the ablation study revealed that nucleotide frequencies provided the most significant improvement in PCC when added to the model, suggesting potential areas for further feature refinement.
  • Data Leakage Mitigation: The paper's contribution of a new data-splitting methodology directly addresses a limitation in previous work, aiming to prevent inflated performance metrics and ensure more reliable evaluations.

Summary updated

Abstract

Abstract The clinical adoption of small interfering RNAs (siRNAs) has prompted the development of various computational strategies for siRNA design, from traditional data analysis to advanced machine learning techniques. However, previous studies have inadequately considered the full complexity of the siRNA silencing mechanism, neglecting critical elements such as siRNA positioning on mRNA, RNA base-pairing probabilities, and RNA-AGO2 interactions, thereby limiting the insight and accuracy of existing models. Here, we introduce siRNADesign , a Graph Neural Network (GNN) framework that leverages both non-empirical and empirical rules-based features of siRNA and mRNA to effectively capture the complex dynamics of gene silencing. In multiple internal datasets, siRNADesign achieves state-of-the-art performance. Significantly, siRNADesign also outperforms existing methodologies in in vitro wet lab experiments and an externally validated dataset. Additionally, we develop a new data-splitting methodology that addresses the data leakage issue, a frequently overlooked issue in previous studies, ensuring the robustness and stability of our model under various experimental settings. Through rigorous testing, siRNADesign has demonstrated remarkable predictive accuracy and robustness, making significant contributions to the field of gene silencing. Furthermore, our approach in redefining data-splitting standards aims to set new benchmarks for future research in the domain of predictive biological modeling for siRNA.
BibTeX
@misc{long2024sirnadesign,
  author = {Rongzhuo Long and Ziyu Guo and Da Han and Xudong Yuan and Guangyong Chen and Pheng‐Ann Heng and Liang Zhang},
  title = {{siRNADesign: A Graph Neural Network for siRNA Efficacy Prediction via Deep RNA Sequence Analysis}},
  year = {2024},
  doi = {10.1101/2024.04.28.591509},
  howpublished = {bioRxiv},
}
Figure