Deciphering gene regulation from gene expression dynamics using deep neural network

Jingxiang Shen, Mariela D. Petkova, Yuhai Tu, Feng Liu, Chao Tang

This paper, published by researchers from Peking University, IBM, and Princeton, presents a novel way to use Deep Neural Networks (DNNs) not just to predict biological data, but to actually "reverse engineer" the underlying logic of life.

TL;DR

The researchers developed a recurrent deep neural network that learns to simulate gene expression dynamics. By performing "in-silico mutagenesis"—systematically blocking links within the trained model—they successfully uncovered the hidden regulatory "wiring diagrams" of complex biological systems, including the spatial patterning of fruit fly embryos.

What Problem Does This Paper Address?

One of the "Holy Grails" of biology is understanding the Gene Regulatory Network (GRN): the complex web of interactions where genes turn each other on or off to perform functions. Traditionally, discovering these networks requires thousands of painstaking "knockout" experiments (deleting a gene and seeing what breaks).

Computationally, this is a nightmare. Simple models often fail to capture the messy nonlinearity of biology, while complex models (like systems of differential equations) have so many parameters that they "overfit"—they memorize the training data perfectly but fail to predict anything new. The authors sought a way to use the massive fitting power of DNNs while making the resulting "black box" interpretable to human scientists.

Methods

The team implemented a Recurrent DNN architecture. Unlike standard "black box" classifiers, this model is designed to be Markovian: the state of the system at time $t + dt$ depends only on the state at time $t$. This mimics the physical reality of biochemical reactions.

The real innovation is their method for interpreting the model:

  1. Training as a Simulator: The DNN is trained on gene expression time-series data until it behaves like an accurate simulator of the biological system.
  2. Link Blocking (In-silico Mutagenesis): To find out if Gene A regulates Gene B, the researchers manually set the influence of Gene A to zero within the trained DNN. They then run the simulation and observe the effect on Gene B. If Gene B's expression increases, Gene A was an inhibitor; if it decreases, Gene A was an activator.
  3. Network Sparsening: They used a "greedy" algorithm to prune redundant links, stripping the network down to its "minimal core" to see which interactions are truly essential for the biological function.

Datasets

The authors validated their approach on two distinct systems:

  • Biochemical Adaptation: A synthetic, two-node system used to see if the DNN could find classic motifs like negative feedback loops.
  • Drosophila Gap-Gene Patterning: A famous and complex real-world system where maternal proteins (Bicoid, Caudal, Torso) trigger a cascade of "gap genes" (Hunchback, Krüppel, Knirps, Giant) to divide the fruit fly embryo into segments. They used protein expression data from the Gregor and Reinitz labs, including wild-type dynamics and a few specific mutant snapshots.

Key Insights and Ideas

  • DNNs as "Implicit Self-Adapting Models": Surprisingly, DNNs seem to resist overfitting in biological contexts. They have a "huge fitting power" but naturally find smooth, monotonic functions that represent biological reality better than over-parameterized differential equations.
  • Robustness is Learnable: The authors found that DNNs are particularly good at learning "robust" mechanisms. In a test against an oversimplified "artificial ground truth," the DNN actually performed better than the ground truth because it inherently favored more robust, biologically-plausible solutions.
  • Sloppy Modes: The paper introduces the concept of "sloppy modes" in network topology. Often, multiple different "wiring" configurations can achieve the same result. The DNN finds a functional solution, and the "link blocking" technique helps distinguish between redundant "fake" activations and the true core topology.
  • Insensitivity to Missing Components: The model is remarkably robust; if a gene is missing from the dataset, the DNN effectively "absorbs" its role into the remaining network, still producing accurate phenotypic predictions.

Key Results

  • High Predictive Accuracy: In the Drosophila gap-gene task, the model correctly predicted 152 out of 166 features (peaks and boundaries of gene expression) across 13 different mutants it had never seen during training.
  • Spatial Precision: The Root-Mean-Square Error (RMSE) for boundary positions was only 4% of the embryo length, a level of accuracy the authors claim surpasses previous differential-equation-based models.
  • Network Discovery: The regulatory network uncovered by the DNN was "strikingly similar" to the "real" network painstakingly inferred by biologists over decades, correctly identifying mutual inhibitions and the directional shifts of gene domains.

Limitations and Future Work

The authors acknowledge a few key hurdles:

  • Data Consistency: Biological data from different labs is often noisy or "inconsistent." The model requires careful preprocessing to ensure different datasets speak the same "language."
  • Artifacts of Simplification: The model showed an unusually strong reliance on "self-activation" for all genes. The authors believe this is a mathematical artifact caused by ignoring the time delay between mRNA transcription and protein translation.
  • The "Black Box" remains: While link blocking provides a window into the model, the thousands of internal weights of the DNN are still not directly readable as biochemical constants.

The next step is applying this "interpretable DNN" approach to larger, less-understood genetic systems where the ground truth isn't already known, potentially accelerating the discovery of new drug targets or synthetic biology circuits.

Summary updated

Abstract

Complex biological functions are carried out by the interaction of genes and proteins. Uncovering the gene regulation network behind a function is one of the central themes in biology. Typically, it involves extensive experiments of genetics, biochemistry and molecular biology. In this paper, we show that much of the inference task can be accomplished by a deep neural network (DNN), a form of machine learning or artificial intelligence. Specifically, the DNN learns from the dynamics of the gene expression. The learnt DNN behaves like an accurate simulator of the system, on which one can perform in-silico experiments to reveal the underlying gene network. We demonstrate the method with two examples: biochemical adaptation and the gap-gene patterning in fruit fly embryogenesis. In the first example, the DNN can successfully find the two basic network motifs for adaptation - the negative feedback and the incoherent feed-forward. In the second and much more complex example, the DNN can accurately predict behaviors of essentially all the mutants. Furthermore, the regulation network it uncovers is strikingly similar to the one inferred from experiments. In doing so, we develop methods for deciphering the gene regulation network hidden in the DNN "black box". Our interpretable DNN approach should have broad applications in genotype-phenotype mapping.
BibTeX
@misc{shen2018deciphering,
  author = {Jingxiang Shen and Mariela D. Petkova and Yuhai Tu and Feng Liu and Chao Tang},
  title = {{Deciphering gene regulation from gene expression dynamics using deep neural network}},
  year = {2018},
  eprint = {1807.08414},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1807.08414},
}
Figure