Sequence-based bacterial small RNAs prediction using ensemble learning strategies

Guifeng Tang, Jingwen Shi, Wenjian Wu, Xiang Yue, Wen ZhangBMC Bioinformatics 2018 · DOI

TL;DR

This paper introduces two novel ensemble learning methods, WAEM and NNEM, for predicting bacterial small RNAs (sRNAs) using a variety of sequence-derived features. These methods significantly outperform existing state-of-the-art approaches, offering a more robust and accurate way to identify these crucial regulatory molecules.

What Problem Does This Paper Address?

Bacterial small non-coding RNAs (sRNAs) are vital regulators involved in numerous cellular processes, from growth to metabolism. Accurately identifying these sRNAs is crucial for understanding bacterial biology and developing new therapeutic strategies. Traditional experimental methods for sRNA identification are laborious and expensive, making computational approaches essential. While various computational methods exist, they often rely on limited features or simpler models. This paper aims to improve sRNA prediction accuracy and robustness by leveraging a diverse set of sequence-derived features and employing advanced ensemble learning strategies.

Methods

The core of this work lies in the development and application of two ensemble learning strategies:

  • Weighted Average Ensemble Method (WAEM): This method combines the predictions of multiple "base predictors," each trained on a specific sequence-derived feature. The final prediction is a weighted sum of these individual predictions. The weights are optimized using a genetic algorithm (GA) to maximize the Area Under the Curve (AUC) score on the training data.
  • Neural Network Ensemble Method (NNEM): This approach uses a multi-layer perceptron (MLP) to integrate features. It involves a two-step process: first, four separate MLP models are trained on merged feature groups (spectrum profile, mismatch profile, reverse compliment k-mer, and pseudo nucleotide composition). The outputs of these models then serve as inputs to a final MLP that generates the overall prediction.

The paper also explores various sequence-derived features, including:

  • Spectrum profile (k-mer profile): Counts of contiguous subsequences of length k.
  • Mismatch profile (k, m)-mismatch profile: Similar to spectrum profile but allows up to 'm' mismatches.
  • Reverse compliment k-mer (k-RevcKmer): Considers the reverse complement of subsequences.
  • Pseudo nucleotide composition features: Captures occurrences of di- and tri-nucleotides along with their physicochemical properties.

Datasets

The study utilizes datasets derived from the experimentally validated sRNAs of Salmonella Typhimurium LT2 (SLT2).

  • Positive instances: 182 experimentally verified sRNAs.
  • Negative instances: Generated by randomly shuffling the genome sequence and extracting fragments of the same length and position as the positive instances.
  • Benchmark Datasets: One balanced dataset (1:1 ratio of positive to negative instances) and four imbalanced datasets (ratios of 1:2, 1:3, 1:4, and 1:5) were created to assess model performance under varying data distributions.

Key Insights and Ideas

  • Feature Diversity is Key: The study strongly suggests that integrating a wide array of sequence-derived features (17 in total) is crucial for achieving high prediction accuracy, as each feature captures different aspects of sRNA sequences.
  • Ensemble Learning's Power: Both WAEM and NNEM demonstrate that ensemble methods are highly effective in combining the strengths of individual feature-based predictors, leading to superior performance compared to single-feature models.
  • NNEM's Superiority: The neural network-based ensemble (NNEM) consistently outperforms the weighted average ensemble (WAEM), indicating that more complex integration strategies can better handle the intricate relationships within the data.
  • Robustness to Imbalance: The proposed methods show remarkable robustness across different dataset imbalance ratios, a critical factor for real-world biological data.
  • Interpretability of WAEM: While NNEM achieves higher accuracy, WAEM offers a degree of interpretability through its optimized weights, revealing which features contribute most to the prediction.

Key Results

  • High AUC Scores: NNEM achieved AUC scores ranging from 0.958 to 0.962 across the five benchmark datasets. WAEM achieved AUC scores from 0.942 to 0.957.
  • Outperforming Existing Methods: Both WAEM and NNEM significantly outperformed existing state-of-the-art methods like Carter's and Barman's, with NNEM showing an average AUC improvement of 2.1% over Barman's method and 69.3% over Carter's method.
  • Feature Importance: Features like 4-spectrum profile (F4), 4-RevcKmer (F12), and PCPseTNC (F15) were identified as having better individual performance, and their contributions were reflected in the higher weights assigned in WAEM.
  • Statistical Significance: A paired t-test confirmed that NNEM's performance was significantly better than WAEM's across all datasets (P-value < 0.05).

Limitations and Future Work

  • Parameter Optimization: While the paper discusses parameter selection for features, the optimization of ensemble model parameters (e.g., GA parameters for WAEM, neural network architecture for NNEM) could be further explored.
  • Dataset Specificity: The study focuses on Salmonella Typhimurium. Future work could involve validating these methods on a broader range of bacterial species to assess their generalizability.
  • Experimental Validation: The authors suggest that their findings can guide wet experiments, implying that experimental validation of predicted sRNAs would be a valuable next step.
  • Exploring More Features: The paper mentions that sequence-derived features bring important information. Investigating even more sophisticated or novel sequence-derived features could further enhance prediction accuracy.

Summary updated

BibTeX
@article{tang2018sequence,
  author = {Guifeng Tang and Jingwen Shi and Wenjian Wu and Xiang Yue and Wen Zhang},
  title = {{Sequence-based bacterial small RNAs prediction using ensemble learning strategies}},
  journal = {BMC Bioinformatics},
  year = {2018},
  doi = {10.1186/s12859-018-2535-1},
}
Figure