Evaluating Between-Pathway Models with Expression Data

Benjamin Hescott, Mark D.m. Leiserson, Thuc Duy Le, Donna K. SlonimJournal of Computational Biology 2010 · DOI

TL;DR

This paper introduces a novel method to validate "Between-Pathway Models" (BPMs) – pairs of potentially redundant biological pathways – by integrating gene expression data from knockout experiments. Their approach, the "ClusterRank Score," effectively identifies compensatory relationships between these pathways, offering a more robust validation than previous methods relying solely on interaction networks.

What Problem Does This Paper Address?

Biological systems often exhibit redundancy, with multiple pathways capable of performing essential functions. "Between-Pathway Models" (BPMs) are network motifs that capture this by identifying pairs of pathways that are hypothesized to be functionally compensatory. Specifically, a BPM is characterized by a pattern of genetic interactions (like synthetic lethality) and physical interactions within and between gene sets representing these pathways. However, validating these hypothesized compensatory relationships has been challenging. Previous methods primarily relied on analyzing protein-protein interaction and genetic interaction networks, often using GO enrichment to infer functional coherence. While this shows genes within a pathway share functions, it doesn't directly prove the compensatory relationship between the two pathways. This paper aims to bridge this gap by using a different type of high-throughput data – gene expression profiles from yeast deletion mutants – to directly assess the functional interplay between proposed compensatory pathways.

Methods

The core of the paper's contribution is the ClusterRank Score, a method designed to measure the coherence of gene expression changes in one pathway (let's call it Y) when a gene in its proposed compensatory pathway (X) is deleted.

Here's how it works:

  1. Data Preparation: They use microarray gene expression data from yeast single-gene deletion strains. For a given deletion mutant (e.g., gene 'g' is deleted), they calculate the "log-10-ratio" of gene expression for all other genes compared to wild-type. This ratio represents the fold change in expression.
  2. Ranking Genes: Genes are ranked based on the absolute value of their log-10-ratio. This is crucial because it captures both strong up-regulation and strong down-regulation, reflecting significant changes in gene expression regardless of direction.
  3. Calculating "Hits" and "Misses": For a target pathway Y, they define "hits" as genes within Y and "misses" as genes not in Y. They then calculate cumulative scores for "hits" and "misses" as they traverse the ranked list of all genes.
  4. ClusterRank Score: The ClusterRank Score is the maximum difference between the cumulative "hit" score and the cumulative "miss" score across all possible ranks. A high positive score suggests that genes in pathway Y tend to be highly ranked (i.e., show strong expression changes) when a gene in the compensatory pathway X is deleted.
  5. Statistical Significance: To assess if a ClusterRank Score is meaningful, they employ a permutation test. They generate 99 random gene sets of the same size as pathway Y and calculate their ClusterRank Scores. The p-value for the observed score is its percentile within this distribution of random scores. A p-value below a chosen threshold (e.g., 0.10) indicates validation.

They also introduce a more stringent random-knockout validation test where they compare the ClusterRank Score of pathway Y against knockouts of random genes not in either pathway X or Y.

Datasets

The primary dataset used is the Rosetta Compendium, which provides genome-wide expression profiles for 276 yeast deletion mutants. This dataset is crucial as it allows for the analysis of gene expression changes upon the deletion of individual genes. The authors also reference and evaluate BPMs previously identified by four different studies:

  • Kelley and Ideker (2005)
  • Ulitsky and Shamir (2007)
  • Ma, Tarone, and Li (2008)
  • Brady et al. (2008)

Key Insights and Ideas

  • Gene expression is a powerful lens for pathway compensation: The core insight is that gene expression data, specifically from knockout experiments, can directly reveal functional relationships between pathways that are not obvious from interaction networks alone. If pathway Y truly compensates for the loss of pathway X, deleting a gene in X should cause coherent changes in the expression of genes in Y.
  • Absolute expression change is key: Using the absolute value of expression changes is critical. It allows the method to detect compensatory pathways where some genes might be up-regulated and others down-regulated in response to a perturbation, as long as the overall response is coherent and strong.
  • BPMs from different studies vary in quality: The study demonstrates that not all identified BPMs are equally well-supported by experimental data. The validation rates differ significantly across the four studies, suggesting that the algorithms used to identify BPMs have varying degrees of success in capturing true compensatory relationships.
  • The "random-knockout" test offers stronger validation: While the permutation test is useful, the random-knockout validation provides a more rigorous assessment of whether a pathway's response is specifically linked to the deletion of a gene in its compensatory pathway, rather than just a general response to any gene deletion.

Key Results

  • The authors successfully validated a subset of BPMs from all four studies using their ClusterRank Score method.
  • The Brady et al. (2008) dataset showed the highest percentage of validated BPMs (24%), suggesting their method for identifying candidate BPMs might be more effective or that their identified pathways are more robustly supported by expression data.
  • Conversely, the Kelley and Ideker (2005) dataset had the lowest validation rate (10%).
  • The paper presents specific examples of validated and unvalidated BPMs, illustrating how the ClusterRank Score and associated plots reveal the coherence (or lack thereof) of gene expression changes. For instance, one validated BPM showed strong, coherent expression changes in the compensatory pathway upon deletion of a gene in the other pathway, while an unvalidated BPM showed scattered and weak responses.
  • Figure 2 shows a heat map of a validated BPM where genes across both pathways exhibit similar expression patterns under different deletion mutants, further supporting their compensatory nature.

Limitations and Future Work

  • Limited knockout data: The primary limitation acknowledged is the incomplete nature of the yeast knockout data. The Rosetta Compendium only covers about 5% of the yeast genome. This means many potential BPMs, especially those involving multiple gene deletions, cannot be fully evaluated.
  • Difficulty validating noisy pathways: With limited data, it's possible to miss validating pathways that are genuinely compensatory but appear "noisy" due to the absence of relevant knockout data.
  • Preference for random-gene-set permutations: Due to the small and potentially biased set of available knockout mutants, the authors preferred random-gene-set permutations over random-mutant-strain permutations for statistical significance testing. They acknowledge that a full set of knockout data would make the random-knockout validation method the preferred approach.
  • Future Work: The authors suggest that as more comprehensive gene expression datasets become available (e.g., through custom array technologies), their method can be applied more broadly. They also envision using their method for pathway refinement, identifying subsets of genes within a pathway that show the most coherent responses, as illustrated in Figure 7. This could lead to more precise pathway models.

Summary updated

Abstract

Between-pathway models (BPMs) are network motifs consisting of pairs of putative redundant pathways. In this article, we show how adding another source of high-throughput data--microarray gene expression data from knockout experiments--allows us to identify a compensatory functional relationship between genes from the two BPM pathways. We evaluate the quality of the BPMs from four different studies, and we describe how our methods might be extended to refine pathways.
BibTeX
@article{hescott2010evaluating,
  author = {Benjamin Hescott and Mark D.M. Leiserson and Thuc Duy Le and Donna K. Slonim},
  title = {{Evaluating Between-Pathway Models with Expression Data}},
  journal = {Journal of Computational Biology},
  year = {2010},
  doi = {10.1089/cmb.2009.0178},
}
Figure