A Hierarchical Bayesian Markovian Model for Motifs in Biopolymer Sequences

Eric P. Xing, Michael I. Jordan, Richard M. Karp, Stuart J. RussellNeurIPS 2002 · OpenReview

TL;DR

This paper introduces a novel hierarchical Bayesian model (HMDM) for identifying motifs in biopolymer sequences. By incorporating prior biological knowledge and modeling positional dependencies within motifs using a hidden Markov model, HMDM significantly improves motif detection sensitivity and accuracy compared to existing methods.

What Problem Does This Paper Address?

Identifying "motifs" in biopolymer sequences (like DNA or proteins) is crucial for understanding biological regulation and function. These motifs are short, recurring patterns that often bind to specific proteins. However, existing computational methods struggle to distinguish true biological motifs from random recurring patterns. A key limitation is that they often assume independence between different positions within a motif, which contradicts biological intuition. Furthermore, they lack a principled way to incorporate existing biological knowledge about motif structures. This paper aims to address these shortcomings by developing a more sophisticated model that captures both the internal structure of motifs and allows for the integration of prior biological information.

Methods

The core of this paper is the Hidden Markov Dirichlet-Multinomial (HMDM) model. Here's a breakdown of its key components:

  • Hierarchical Structure: The model is hierarchical, meaning it has multiple layers of probabilistic modeling.
  • Dirichlet-Multinomial Mixture: At each position within a motif, the nucleotide (or amino acid) distribution is modeled by a multinomial distribution. Crucially, these multinomial distributions are not fixed but are drawn from a mixture of Dirichlet distributions. Each Dirichlet distribution represents a "prototype" pattern for a motif position. This allows for multi-modal distributions at a given position, reflecting biological reality.
  • Hidden Markov Model (HMM) for Transitions: The choice of which Dirichlet prototype to use at each position within a motif is governed by a first-order Hidden Markov Model. This HMM captures the positional dependencies – how the pattern at one position influences the pattern at the next. This is a key innovation, moving beyond the independence assumption of previous models.
  • Variational Bayesian Learning: Fitting such a complex model to data is challenging. The authors employ a variational EM algorithm within an empirical Bayesian framework. This approach approximates the intractable posterior distributions with simpler ones, allowing for efficient parameter learning and inference. Variational inference is also used for motif detection.

What's new here is the combination of a Dirichlet-mixture model for position-specific distributions with an HMM to model the transitions between these Dirichlet components. This allows the model to learn and represent complex, multi-modal patterns and their dependencies.

Datasets

The primary dataset used for training and evaluation is the Promoter Database of Saccharomyces cerevisiae (SCPD). This dataset contains twenty known motifs, each with 6 to 32 experimentally identified instances. The authors also mention using a "decoy" dataset, which consists of permuted versions of the true motifs, to test the model's ability to distinguish genuine motifs from random patterns.

Key Insights and Ideas

  • Positional Dependencies Matter: The HMDM's ability to model transitions between motif "states" (represented by Dirichlet distributions) is a significant improvement. This allows it to capture biologically relevant patterns like "site clustering," where conserved and less conserved positions tend to group together.
  • Prior Knowledge Integration: The Dirichlet prior allows for the incorporation of biological intuition. For example, the model can learn that some positions are highly specific (favoring a single nucleotide) while others are more variable.
  • Multi-modality is Key: Real motifs often exhibit multiple distinct patterns at a given position. The Dirichlet-mixture approach effectively captures this multi-modality, which simpler models miss.
  • Modular Design: The HMDM framework is designed to be modular, allowing for separate modeling of motif patterns and motif instance distributions. This makes it easier to extend the model to more complex scenarios, like modeling multiple motifs or their interactions.

Key Results

  • Improved Sensitivity and Specificity: The HMDM model demonstrated significantly higher sensitivity in detecting motifs compared to standard Position Weight Matrix (PWM) models (which are essentially MEME-like).
  • Distinguishing True Motifs: The model showed a notable ability to distinguish genuine motifs from false recurring patterns, especially when tested against permuted "decoy" motifs.
  • Capturing Biological Patterns: The learned Dirichlet components and transition probabilities aligned well with biological expectations, such as identifying "homogeneous" (single nucleotide favored) and "heterogeneous" (more uniform) prototypes and confirming the site clustering property.
  • Performance on SCPD Dataset: In motif finding tasks on the SCPD dataset, HMDM models (including a mixture of HMDMs) generally outperformed the PM model, particularly in scenarios with true motifs and decoys.

Limitations and Future Work

  • Computational Complexity: While variational inference makes learning tractable, complex models can still be computationally intensive.
  • "Atypical" Motifs: The paper notes that very short or highly conserved motifs (like "mcb") can be challenging for HMM-based models, potentially due to limited positional information or atypical structure.
  • Future Work: The authors suggest several avenues for future research:
  • Developing more expressive global models for motif occurrences (beyond the "one-per-sequence" assumption).
  • Extending the model to capture dependencies between multiple motifs.
  • Investigating more sophisticated HMM structures or alternative transition models.
  • Exploring the application of HMDM to other sequence modeling problems beyond DNA motifs.

Summary updated

Abstract

We propose a dynamic Bayesian model for motifs in biopolymer se- quences which captures rich biological prior knowledge and positional dependencies in motif structure in a principled way. Our model posits that the position-specific multinomial parameters for monomer distribu- tion are distributed as a latent Dirichlet-mixture random variable, and the position-specific Dirichlet component is determined by a hidden Markov process. Model parameters can be fit on training motifs using a vari- ational EM algorithm within an empirical Bayesian framework. Varia- tional inference is also used for detecting hidden motifs. Our model im- proves over previous models that ignore biological priors and positional dependence. It has much higher sensitivity to motifs during detection and a notable ability to distinguish genuine motifs from false recurring patterns.
BibTeX
@inproceedings{xing2002hierarchical,
  author = {Eric P. Xing and Michael I. Jordan and Richard M. Karp and Stuart J. Russell},
  title = {{A Hierarchical Bayesian Markovian Model for Motifs in Biopolymer Sequences}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2002},
  url = {https://openreview.net/forum?id=BkXLTL-dZS},
}
Figure