Mixture of experts models to exploit global sequence similarity on biomolecular sequence labeling

Cornelia Caragea, Jivko Sinapov, Drena Dobbs, Vasant G. HonavarBMC Bioinformatics 2009 · DOI · OpenReview

Introduction

In bioinformatics, sequence labeling is a fundamental task involving the prediction of functional properties for individual residues within a protein. Identifying interface residues that bind to RNA or DNA is a primary application. Traditionally, these predictions rely on a "local window" approach, where the algorithm examines a small neighborhood of residues around the target. However, functional sites are highly diverse; proteins with entirely different local sequences may bind to the same target. This research proposes that global evolutionary context—the "big picture" of a protein's overall structure and family—is a critical missing component in standard models.

Methodology: Hierarchical Mixture of Experts (HME)

To integrate global similarity without sacrificing local precision, the authors developed a Hierarchical Mixture of Experts (HME) framework. This approach moves away from a single "one-size-fits-all" classifier in favor of a specialized tree structure:

  1. Global Similarity Matrix: Pairwise similarities between all sequences are calculated using the Needleman-Wunsch global alignment algorithm.
  2. Spectral Clustering: The similarity matrix is used to perform recursive 2-way spectral clustering, organizing sequences into a hierarchical tree where globally similar proteins reside in the same leaf nodes.
  3. Expert Classifiers: Specialized "local experts" (either Naïve Bayes or Logistic Regression) are trained specifically on the sequences within each cluster.
  4. Bayesian Gating: For new sequences, internal gating networks determine the weight assigned to each expert's output, allowing for "soft partitioning" where a sequence can be influenced by multiple experts.

Experimental Results

The model was evaluated on RNA-protein (435 complexes) and DNA-protein (1,259 complexes) interface prediction tasks. To test robustness, the authors used non-redundant datasets with identity cutoffs ranging from 30% to 90%.

Precision-Recall curves comparing the Mixture of Naive Bayes-global model against standard Naive Bayes and ensemble baselines.

As shown in the Precision-Recall analysis, the MoE models consistently outperformed both baseline classifiers and traditional ensembles. On the challenging 30% identity RNA-protein dataset, the MoE model achieved superior precision across various recall levels compared to the standard Naïve Bayes baseline.

Correlation Coefficient performance across varying sequence redundancy cutoffs (30% to 90%), demonstrating the robustness of the MoE approach.

Furthermore, the performance gains were maintained across all levels of sequence redundancy. The Correlation Coefficient (CC) for the MoE model remained consistently higher than the baseline, proving that the framework effectively learns robust patterns rather than simply memorizing closely related sequences.

Discussion and Conclusion

The success of the HME framework highlights that global similarity is a more effective grouping mechanism than local sequence patterns for functional site prediction. By splitting the protein universe into specialized neighborhoods, the model can learn multiple "sub-consensuses" for binding sites that would otherwise be lost in a global average.

Future work may involve jointly learning the tree structure and classifiers or exploring 3D structural similarity measures to further refine the grouping of experts. Despite the computational cost of calculating full similarity matrices for large datasets, the MoE approach offers a significant leap in accuracy for complex biomolecular sequence labeling tasks.

Summary updated

Abstract

Background Identification of functionally important sites in biomolecular sequences has broad applications ranging from rational drug design to the analysis of metabolic and signal transduction networks. Experimental determination of such sites lags far behind the number of known biomolecular sequences. Hence, there is a need to develop reliable computational methods for identifying functionally important sites from biomolecular sequences. Results We present a mixture of experts approach to biomolecular sequence labeling that takes into account the global similarity between biomolecular sequences. Our approach combines unsupervised and supervised learning techniques. Given a set of sequences and a similarity measure defined on pairs of sequences, we learn a mixture of experts model by using spectral clustering to learn the hierarchical structure of the model and by using bayesian techniques to combine the predictions of the experts. We evaluate our approach on two biomolecular sequence labeling problems: RNA-protein and DNA-protein interface prediction problems. The results of our experiments show that global sequence similarity can be exploited to improve the performance of classifiers trained to label biomolecular sequence data. Conclusion The mixture of experts model helps improve the performance of machine learning methods for identifying functionally important sites in biomolecular sequences.
BibTeX
@article{caragea2009mixture,
  author = {Cornelia Caragea and Jivko Sinapov and Drena Dobbs and Vasant G. Honavar},
  title = {{Mixture of experts models to exploit global sequence similarity on biomolecular sequence labeling}},
  journal = {BMC Bioinformatics},
  year = {2009},
  doi = {10.1186/1471-2105-10-s4-s4},
}
Figure