Learning sparse models for a dynamic Bayesian network classifier of protein secondary structure

Zafer Aydin, Ajit P. Singh, Jeff A. Bilmes, William Stafford NobleBMC Bioinformatics 2011 · DOI

TL;DR

This paper introduces a novel method for predicting protein secondary structure by combining Dynamic Bayesian Networks (DBNs) with Support Vector Machines (SVMs). Crucially, they developed an algorithm to create "sparse" DBNs, significantly reducing model complexity and computational cost while maintaining high prediction accuracy and revealing biologically relevant correlations.

What Problem Does This Paper Address?

Predicting a protein's 3D structure is a fundamental challenge in biology, and protein secondary structure prediction (e.g., identifying helices, strands, and loops) is a critical intermediate step. While existing methods using DBNs and SVMs achieve state-of-the-art accuracy, they can be computationally expensive and prone to overfitting due to the large number of parameters in their models. This paper aims to address this by developing a method that not only achieves high accuracy but also creates more interpretable and efficient models by learning sparse representations.

Methods

The core of the paper lies in two main contributions:

  • Combined DBN-SVM Classifier: They integrate a Dynamic Bayesian Network (DBN) with a Support Vector Machine (SVM). The DBN models the sequential nature of protein sequences, while the SVM combines the outputs of multiple DBNs and other features (like PSSMs) to make the final prediction.
  • Sparse DBN Learning Algorithm: They introduce an iterative algorithm that interleaves Expectation-Maximization (EM) with a sparsification step. This algorithm effectively prunes less important parameters (edges in the DBN's graphical model), leading to sparse models. This process aims to remove redundant parameters without sacrificing predictive accuracy.

Datasets

The authors used several standard datasets for training and evaluation:

  • CB513: A well-established and difficult benchmark dataset for secondary structure prediction.
  • SD576: Another benchmark dataset used for comparing their method with existing DBN approaches.
  • PDB-PC15: A dataset specifically curated for analyzing non-local correlations in beta-strands.

Key Insights and Ideas

  • Sparsity as a Feature: The paper demonstrates that sparsity in DBNs isn't just about efficiency; it's a powerful tool for uncovering biologically meaningful correlations within protein sequences.
  • Balancing Complexity and Accuracy: The sparsification algorithm successfully navigates the trade-off between model complexity and predictive accuracy, showing that many parameters in DBNs are indeed redundant.
  • Interpretability through Sparsity: By identifying which "dlinks" (parameters) remain in the sparse model, the authors can gain insights into local and non-local interactions that are characteristic of different secondary structure elements.
  • DBN-SVM Synergy: Combining DBNs with SVMs, especially when using diverse feature representations (like PSI-BLAST and HHMAKE PSSMs), significantly boosts predictive performance.

Key Results

  • State-of-the-Art Accuracy: Their method achieved a per-residue accuracy (Q3) of 80.3% on the CB513 benchmark, comparable to the best-performing methods at the time.
  • Significant Sparsity Achieved: They demonstrated that their algorithm can remove 70-95% of DBN parameters while maintaining predictive accuracy.
  • Speedup: At 90% sparsity, predictions were computed three times faster than a fully dense model.
  • Biological Correlations: The sparse models successfully identified known local correlations (e.g., in helices and beta-strands due to hydrogen bonds) and provided evidence for the importance of non-local interactions in beta-strands.
  • Improved Performance: Their combined DBN-SVM approach outperformed existing DBN methods by a statistically significant margin.

Limitations and Future Work

  • Computational Cost of Optimization: While the resulting sparse models are fast, the process of learning and optimizing hyperparameters for the DBNs and SVMs can be computationally intensive.
  • Focus on Local vs. Non-local: While they explored non-local interactions in beta-strands, the paper suggests further work is needed to fully capture these complex relationships.
  • Future Directions: The authors propose extending their sparse modeling paradigm to include additional types of observations, identify even longer-range correlations, and apply sparse models to other bioinformatics tasks like contact map prediction and 3D structure prediction. They also suggest using the identified non-local interaction patterns as features for other prediction algorithms.

Summary updated

BibTeX
@article{aydin2011learning,
  author = {Zafer Aydin and Ajit P. Singh and Jeff A. Bilmes and William Stafford Noble},
  title = {{Learning sparse models for a dynamic Bayesian network classifier of protein secondary structure}},
  journal = {BMC Bioinformatics},
  year = {2011},
  doi = {10.1186/1471-2105-12-154},
}
Figure