Systems biology informed deep learning for inferring parameters and hidden dynamics

Alireza Yazdani, Lu Lu, Maziar Raissi, George Em KarniadakisPLOS Computational Biology 2020 · OpenReview

TL;DR

This paper introduces a novel "systems-informed neural network" (SINN) that embeds the governing ordinary differential equations (ODEs) of biological systems directly into the deep learning architecture. This approach allows for the accurate inference of unknown parameters and the prediction of hidden dynamics, even with sparse and noisy experimental data, offering a powerful new tool for systems biology.

What Problem Does This Paper Address?

Systems biology aims to understand complex biological processes by modeling them with systems of ordinary differential equations (ODEs). A major hurdle in this field is accurately estimating the numerous unknown parameters within these ODEs and predicting the behavior of species that are not directly observed experimentally. Traditional methods often struggle with limited, noisy, or sparse data, leading to issues of parameter non-identifiability and making it difficult to fully decipher system dynamics. This paper tackles the challenge of inferring both unknown parameters and unobserved dynamics from such limited experimental information.

Methods

The core innovation is the development of a "systems-informed neural network" (SINN). This architecture integrates the system of ODEs directly into the neural network's loss function. Specifically, the residuals of the ODEs are added to the loss, effectively constraining the network's output to satisfy the underlying biological equations. This is achieved through a deep learning framework that includes:

  • Input-scaling layer: To normalize the input time variable.
  • Feature layer: To incorporate domain-specific patterns (e.g., periodicity, decay) into the network's input.
  • Fully-connected layers: Standard neural network layers for learning.
  • Output-scaling layer: To scale the network's output to match the expected magnitudes of the state variables.

The key novelty lies in the physics-informed aspect, where the ODEs act as a form of regularization, making the network robust to noisy and sparse data. The training process optimizes both the neural network's parameters and the ODE model's parameters simultaneously. A two-stage training strategy is employed: first, supervised losses (data and auxiliary) are used to quickly match observed data, followed by training with all losses (including the unsupervised ODE loss) for convergence.

Datasets

The paper utilizes synthetic datasets generated from established biological models for evaluation:

  • Yeast glycolysis model: A benchmark for nonlinear dynamics.
  • Cell apoptosis model: A model known for parameter non-identifiability issues.
  • Ultradian glucose-insulin model: Used for forecasting glucose levels based on nutritional intake.

The authors simulate noisy and noiseless observations for these models, often using only a few observable species to mimic real-world experimental limitations.

Key Insights and Ideas

  • ODE as a Regularizer: Embedding ODEs into the loss function acts as a powerful regularization technique, preventing overfitting to noisy data without requiring explicit L1 or L2 regularization.
  • Inferring Hidden Dynamics: The SINN can successfully predict the behavior of unobserved species, providing a more complete picture of the system's dynamics than direct observation allows.
  • Parameter Inference from Sparsity: The method demonstrates the ability to infer unknown model parameters even when data is scarce (e.g., using only one observable species).
  • Handling Non-Identifiability: The approach shows promise in inferring parameters in systems that are known to have structural or practical non-identifiability issues, though careful analysis is still needed.
  • Inferring Hidden Forcing: A significant advantage is the ability to infer unknown external drivers or forcing functions within a model, as demonstrated with the nutritional intake in the glucose-insulin model.

Key Results

The SINN demonstrated strong performance across three benchmark problems:

  • Yeast Glycolysis: Excellent agreement between inferred and exact dynamics for all species, even with noisy observations. Key parameters were identified with reasonable accuracy.
  • Cell Apoptosis: The model successfully inferred dynamics for most species using only one observable, despite the system's known non-identifiability challenges. Four parameters were identified with relatively high accuracy.
  • Ultradian Glucose-Insulin: The algorithm accurately inferred model parameters and forecasted glucose levels following simulated nutritional intake events, even when the timing and quantity of these events were unknown.

Limitations and Future Work

  • Identifiability Analysis: While the paper uses identifiability analysis (FIM, null eigenvectors) to interpret results, it acknowledges that FIM has limitations, especially for nonlinear and partially observed systems. More advanced methods are suggested for future work.
  • Computational Cost: The paper mentions that inferring all parameters and hidden dynamics can be challenging for very large systems.
  • Data Requirements: While the method excels with sparse data, the authors suggest that increasing the number of observed species could help resolve structural non-identifiabilities.
  • Future Directions: The authors plan to apply the method to larger, real-world biological systems and datasets, moving beyond synthetic benchmarks. They also aim to further explore the identifiability analysis and its implications.

Summary updated

Abstract

Author summary The dynamics of systems biological processes are usually modeled using ordinary differential equations (ODEs), which introduce various unknown parameters that need to be estimated efficiently from noisy measurements of concentration for a few species only. In this work, we present a new “systems-informed neural network” to infer the dynamics of experimentally unobserved species as well as the unknown parameters in the system of equations. By incorporating the system of ODEs into the neural networks, we effectively add constraints to the optimization algorithm, which makes the method robust to noisy and sparse measurements.
BibTeX
@article{yazdani2020systems,
  author = {Alireza Yazdani and Lu Lu and Maziar Raissi and George Em Karniadakis},
  title = {{Systems biology informed deep learning for inferring parameters and hidden dynamics}},
  journal = {PLOS Computational Biology},
  year = {2020},
  url = {https://openreview.net/forum?id=S4gEQfHNz7c},
}
Figure