Simulation-Based Methods for Optimal Sampling Design in Systems Biology

Tuan Minh Ha, Binh Thanh Nguyen, Lam Si Tung HoKse 2024 · DOI

TL;DR

This paper introduces two novel simulation-based methods, E-optimal-ranking (EOR) and Attention-based LSTM (At-LSTM), for determining optimal sampling times in systems biology. These methods overcome the limitations of traditional approaches by not requiring an initial estimate of model parameters, leading to more robust and accurate parameter estimation.

What Problem Does This Paper Address?

In systems biology, understanding complex biological systems often involves building dynamic models described by differential equations. To accurately estimate the parameters of these models, researchers collect data through experiments. However, conducting these experiments and collecting data can be time-consuming, labor-intensive, and costly. Therefore, the crucial task of selecting when to take samples (optimal sampling design) is paramount. Traditional methods for optimal experimental design (OED) often rely on the Fisher Information Matrix (FIM) and criteria like A-, D-, and E-optimality. A significant drawback of these classical methods is their requirement for an initial estimate of the model parameters. If this initial estimate is inaccurate, the resulting sampling design can be suboptimal, leading to inefficient experiments and less accurate parameter estimates. This paper addresses this critical gap by proposing simulation-based OED methods that bypass the need for an initial parameter estimate.

Methods

The paper proposes two simulation-based methods for optimal sampling design:

  • E-optimal-ranking (EOR): This method adapts the classical E-optimal criterion. Instead of relying on a single initial parameter estimate, EOR simulates a large number of parameter vectors uniformly from the parameter space. For each simulated parameter set, it calculates the E-optimal criterion (maximizing the smallest eigenvalue of the FIM). The sampling times are then ranked based on these simulated E-optimal values. By averaging these ranks over many simulations, EOR identifies the sampling times that consistently yield high E-optimality across a range of possible parameter values. This approach effectively accounts for parameter uncertainty without requiring a specific initial guess.
  • Attention-based Long Short-Term Memory (At-LSTM): This method leverages deep learning. First, it simulates data from the dynamic system using uniformly sampled parameter sets. Then, an Attention-based LSTM (At-LSTM) neural network is trained to map the simulated data to the corresponding parameters. The attention mechanism within the LSTM allows it to identify which time points are most influential in predicting the parameters. By extracting these attention weights, the method identifies the sampling times that provide the most information for parameter estimation.

Datasets

The paper does not introduce new datasets. Instead, it uses simulated data generated from two well-established models in systems biology for evaluation:

  • Lotka-Volterra Model: A classic prey-predator model used to study population dynamics.
  • Three-compartment Pharmacokinetic Model: A model commonly used to describe the absorption, metabolism, and clearance of drugs in the human body.

For each model, the authors simulate 1000 datasets by sampling parameters uniformly from their respective parameter spaces and then generating observations at 101 equally spaced time points. These simulated datasets are then used to evaluate the performance of the proposed sampling design methods against classical and random methods.

Key Insights and Ideas

  • Eliminating the Need for Initial Parameter Estimates: The core innovation is developing OED methods that are robust to parameter uncertainty by not requiring an initial guess. This significantly broadens the applicability of OED in real-world scenarios where initial parameter estimates might be poor or unavailable.
  • Leveraging Simulation for Robustness: Both proposed methods use simulation to explore the parameter space. EOR uses simulation to average optimality across parameter distributions, while At-LSTM uses simulation to train a model that learns informative sampling times.
  • Deep Learning for Feature Importance: The At-LSTM method demonstrates the potential of deep learning, specifically attention mechanisms, to identify critical time points for data collection in dynamic systems. This offers a data-driven approach to OED.
  • Performance Gains: The simulation studies show that the proposed methods consistently outperform naive random sampling and the classical E-optimal criterion, especially when initial parameter estimates might be inaccurate.

Key Results

The paper presents quantitative results comparing the average error of parameter estimation using sampling times selected by Random, E-optimal, EOR, and At-LSTM methods.

  • Lotka-Volterra Model: The EOR and At-LSTM methods achieved significantly lower average estimation errors compared to Random and E-optimal methods. The violin plots (Figure 2) visually illustrate this, showing smaller error distributions for EOR and At-LSTM. Tukey's HSD test confirmed that EOR and At-LSTM were statistically significantly better than Random and E-optimal, with EOR and At-LSTM performing similarly to each other.
  • Three-compartment Pharmacokinetic Model: For this model, the EOR and E-optimal methods showed similar performance and were better than Random and At-LSTM. However, the paper notes that the performance of At-LSTM can be sensitive to tuning. Tukey's HSD test indicated that EOR and At-LSTM were significantly better than Random, and E-optimal was also better than Random. EOR and At-LSTM were similar, and E-optimal and At-LSTM were also similar.

Limitations and Future Work

  • At-LSTM Sensitivity: The authors acknowledge that the performance of the At-LSTM method can be sensitive to hyperparameter tuning.
  • Exploring Other ML Models: They suggest that replacing LSTM with other machine learning models in the At-LSTM framework could lead to new and potentially improved methods.
  • Ensemble Methods: The paper proposes exploring the possibility of combining several optimal sampling schemes using ensemble methods for even better performance.
  • Real-world Data: While simulations are valuable, future work could involve validating these methods on real experimental data from biological systems.

Summary updated

BibTeX
@inproceedings{ha2024simulation,
  author = {Tuan Minh Ha and Binh Thanh Nguyen and Lam Si Tung Ho},
  title = {{Simulation-Based Methods for Optimal Sampling Design in Systems Biology}},
  booktitle = {Kse},
  year = {2024},
  doi = {10.1109/kse63888.2024.11063540},
}
Figure