without Explicitly Constructing Gene Regulatory Networks

IJCAI 2013

TL;DR

This paper introduces a novel approach to controlling gene regulatory networks (GRNs) by directly using gene expression data with Batch Reinforcement Learning (Batch RL). This method bypasses the computationally expensive step of building a GRN model, enabling control policies for thousands of genes in seconds, achieving comparable results to existing model-dependent methods.

What Problem Does This Paper Address?

Controlling gene regulatory networks (GRNs) is crucial for understanding cellular behavior and developing therapies. GRNs model the complex interactions between genes. A major challenge in controlling these networks is the computational complexity of existing methods. Many approaches rely on first constructing a model of the GRN, such as a Probabilistic Boolean Network (PBN), and then deriving a control policy from this model. However, building these PBNs is extremely time-consuming and resource-intensive, making them infeasible for systems with more than a few dozen genes. This limitation hinders the application of control strategies to larger, more realistic biological systems.

Methods

The core innovation of this paper is the application of Batch Mode Reinforcement Learning (Batch RL) to GRN control. Instead of building an explicit model of the GRN, the authors treat time-series gene expression data as a sequence of "experience tuples" (state, action, next state, cost). Batch RL algorithms, specifically Least-Squares Fitted Q Iteration (FQI), are then used to learn an approximate control policy directly from these tuples.

Key aspects of their method include:

  • Data-driven approach: Gene expression data is directly used to learn the policy, eliminating the need for PBN construction.
  • Experience Tuples: Gene expression samples are converted into (state, action, next state, cost) tuples. States are discretized gene expression values. Actions represent interventions on specific "input" genes (e.g., flipping their expression state). Costs are assigned based on whether the system is in a desirable or undesirable state.
  • Least-Squares Fitted Q Iteration (FQI): This algorithm uses parametric approximation (feature vectors and parameter vectors) to learn the state-action function. It iteratively refines a parameter vector using least-squares linear regression on the collected experience tuples.
  • Feature Engineering: Simple yet effective features are used, directly employing the discretized gene expression values of the current state as features.

Datasets

The paper evaluates its method using the melanoma dataset presented in [Bittner et al., 2000]. This dataset contains gene expression profiles. For experiments, they used:

  • A seven-gene subset of the melanoma dataset for initial evaluation.
  • An extended 28-gene subset of the melanoma dataset for large-scale evaluation.

Key Insights and Ideas

  • Bypassing Model Construction: The most significant insight is that a functional control policy can be learned directly from observed system dynamics (gene expression data) without the need for an explicit, computationally expensive GRN model like a PBN.
  • Scalability: Batch RL, by avoiding PBN construction, offers a dramatic improvement in scalability, allowing control policy generation for systems with thousands of genes.
  • Near-Optimal Performance: Despite being a model-free approach, the learned policies achieve solution quality comparable to, and in some cases better than, policies derived from optimal model-based methods.
  • Time Efficiency: The method is orders of magnitude faster than traditional PBN-based approaches, completing policy generation for large systems in seconds.

Key Results

  • Scalability: The method successfully generated control policies for gene regulation systems with several thousands of genes in just several seconds.
  • Performance Comparison (7-gene system): In the melanoma experiment, the Batch RL method shifted the probability mass of desirable states to 0.98, compared to 0.96 for the Value Iteration method. The expected cost was also near-optimal (0.40 vs. 0.21 for Value Iteration).
  • Performance Comparison (28-gene system): For the larger system, the probability sum of desirable states increased from 0.01 (uncontrolled) to 0.8 (controlled by Batch RL), demonstrating significant control.
  • Time Efficiency: For a 28-gene system, the method produced a policy in seconds, whereas PBN construction for systems over 50 genes was infeasible on their hardware.

Limitations and Future Work

  • Feature Engineering: While the current features are effective, the paper acknowledges that different feature sets could be explored.
  • Cost Function Definition: The cost function is defined based on previous studies; exploring alternative cost formulations could be a future direction.
  • Comparison to Online RL: The paper focuses on Batch RL, which uses pre-collected data. Future work could explore online RL approaches for GRN control.
  • Real-world Biological Validation: The results are demonstrated on biological datasets, but direct experimental validation in living organisms is a natural next step.

Summary updated

BibTeX
@inproceedings{2013without,
  title = {{without Explicitly Constructing Gene Regulatory Networks}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2013},
}

Loading related content...

Figure