A benchmark suite of intracellular Boolean model variants and multiscale simulations for computational biology

Marco Masera, Riccardo Smeriglio, Roberta Bardini, Alessandro Savino, Stefano Di CarloarXiv 2026 · arXiv

TL;DR

PhysiBench is a new open-source benchmark resource designed to bridge the gap between systems biology and machine learning by providing 612 executable intracellular Boolean network variants and a massive dataset of 120,000 multiscale stochastic simulations. By standardizing inputs and outputs across diverse biological models, it provides the structured, large-scale data necessary to train fast machine learning emulators (surrogate models) and optimize complex biomedical simulation workflows.

What Problem Does This Paper Address?

In computational systems biology, researchers use multiscale simulations to study how microscopic intracellular signaling scales up to macroscopic cell population dynamics, such as tumor growth or tissue patterning. However, these mechanistic simulations are incredibly expensive to run. To perform optimization tasks—like finding the perfect drug dosing schedule—algorithms must simulate the model thousands of times, creating a massive computational bottleneck.

While fields like climate modeling and physics have standardized machine learning benchmarks (like WeatherBench or PDEBench) to train fast neural network "surrogates" that mimic these simulations at a fraction of the cost, systems biology has lacked an equivalent. Existing repositories share individual biological models but do not provide large, interconnected families of executable models paired with systematically generated simulation trajectories. PhysiBench addresses this fragmentation by offering a controlled, diverse, and reproducible testing ground for scientific machine learning and optimization algorithms.

Methods

The authors developed an automated pipeline to generate a vast, diverse suite of executable models and simulation data from seven established reference Boolean networks (covering cell-cycle control, cancer signaling, immune response, and more).

Figure 2: Overview of the pipeline. Starting from seven reference intracellular Boolean regulatory network models, benchmark suite construction proceeds through three stages: (1) mutation-based generation of candidate models, (2) online behavioral evaluation, and (3) offline sensitivity eval- uation. This produces a benchmark suite of 612 variant intracellular Boolean regulatory network models. A subsequent model selection step retains 60 models to increase coverage of diverse dy- namical regimes for simulation dataset generation. In the second phase, simulations are executed under fixed simulator settings and controlled variations in initial conditions and stimulation pro- tocols, producing 120,000 time-resolved multiscale stochastic simulations.

The pipeline operates in three main stages:

  1. Mutation-Based Generation: The reference models are stochastically mutated (e.g., swapping logic rules, replacing operators, adding nodes) to create structural variants. Crucially, every variant is bound to a standardized interface: exactly one input node (for external stimulation) and three output nodes (governing cell survival, proliferation, and death).
  2. Online Behavioral Filtering: To eliminate redundant models, candidates are simulated under 48 distinct protocols. Models whose population-level behavioral signatures correlate too highly (Pearson correlation > 0.85) with already accepted models are discarded.
  3. Offline Sensitivity Filtering: The remaining models are evaluated across 215 simulation contexts to ensure they actively respond to changes in input parameters, filtering out models that are executable but dynamically inert.

Finally, the authors selected 60 highly diverse models from this suite and ran 2,000 independent simulations per model using the hybrid agent-based framework PhysiBoSS/PhysiCell, varying the stimulation protocols and initial spatial configurations.

Datasets

PhysiBench introduces two primary linked components:

  • Benchmark Model Suite: A collection of 612 executable intracellular Boolean regulatory network variants derived from 7 distinct biological families. They are provided in standard MaBoSS-compatible .bnd and .cfg formats.
  • Multiscale Trajectory Dataset: A 23 GB dataset containing 120,000 time-resolved simulations (2,000 runs for each of the 60 selected models). The data includes full cell-level spatial coordinates ($x, y, z$) and cell-cycle/death-state annotations across time, completely mapped to their respective stochastic seeds and input parameters.
Overview of the PhysiBench resource, highlighting domains, gaps, and tasks.

Key Insights and Ideas

  • Interface Standardization is Powerful: Enforcing a uniform 1-input, 3-output interface across entirely different biological systems allows researchers to test a single machine learning architecture or optimization algorithm across hundreds of distinct dynamical regimes without rewriting code.
  • Synthetic Diversity Trumps Manual Curation: Manually building hybrid multiscale models is incredibly resource-intensive. This paper proves that combining stochastic network mutation with strict behavioral filtering can successfully generate structurally sound, highly diverse networks that maintain traceable lineages to real biological systems.
  • Exceptionally Low Redundancy: The filtering pipeline was highly effective; the average correlation between the output-response profiles of the selected dataset models is just 0.06015, indicating highly independent and diverse dynamic behaviors.

Key Results

  • The pipeline successfully whittled down over 200,000 test candidates to 612 sensitive, non-redundant models.
  • Generating the 120,000-trajectory dataset required ~2,543 CPU-hours on Intel Xeon Gold processors, representing a substantial computational investment that researchers can now bypass.
  • Technical Validation: The structural diversity of the suite was rigorously validated using three graph-based distance metrics (DeltaCon, Ipsen–Mikhailov distance, and Quantum Jensen–Shannon divergence), confirming that the mutation process yielded genuinely distinct regulatory structures rather than minor syntactic variations.

Limitations and Future Work

  • Methodological, Not Clinical: The authors explicitly state that these models are synthetic constructs designed for algorithmic benchmarking. They are not independently calibrated against experimental data and should not be used to predict real-world clinical drug efficacy or toxicity.
  • Boolean Simplification: The discrete, switch-like nature of Boolean logic cannot capture graded signal transduction or continuous dose-response relationships.
  • Dimensionality Constraints: The multiscale simulations are currently restricted to a 2D spatial domain, meaning they omit 3D tissue architecture and complex extracellular matrix mechanics that influence real-world cell behavior. Future extensions could expand these benchmarks into full 3D environments.

Summary updated

Abstract

We present PhysiBench, an open resource for developing and evaluating computational methods in systems biology including a benchmark suite of 612 executable intracellular Boolean regulatory network variants and a dataset of 120,000 time-resolved multiscale stochastic simulations. The benchmark models are derived from seven published Boolean networks spanning cell-cycle control, developmental patterning, cancer signaling, immune response, and cell-fate decisions, and are executable in the PhysiBoSS/PhysiCell multiscale simulation framework. Model variants are generated through mutation-based model construction, online behavioral filtering, and offline sensitivity evaluation. The simulation dataset is produced from 60 selected models under systematically sampled stimulation protocols and fixed model-level initial configurations. Each trajectory is linked to its model identifier, input-parameter file, stochastic seed, and cell-level output file. PhysiBench supports direct simulation, surrogate modeling, data-driven inference, simulation-based optimization, and comparative benchmarking. Technical validation includes file-integrity and executability checks, graph-based structural diversity analyses, and behavioral heterogeneity assessment from multiscale simulation outputs.
BibTeX
@misc{masera2026benchmark,
  author = {Marco Masera and Riccardo Smeriglio and Roberta Bardini and Alessandro Savino and Stefano Di Carlo},
  title = {{A benchmark suite of intracellular Boolean model variants and multiscale simulations for computational biology}},
  year = {2026},
  eprint = {2606.18215},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.18215},
}

Loading related content...

Figure