TL;DR
PhysiBench is a new open-source benchmark resource designed to bridge the gap between systems biology and machine learning by providing 612 executable intracellular Boolean network variants and a massive dataset of 120,000 multiscale stochastic simulations. By standardizing inputs and outputs across diverse biological models, it provides the structured, large-scale data necessary to train fast machine learning emulators (surrogate models) and optimize complex biomedical simulation workflows.
What Problem Does This Paper Address?
In computational systems biology, researchers use multiscale simulations to study how microscopic intracellular signaling scales up to macroscopic cell population dynamics, such as tumor growth or tissue patterning. However, these mechanistic simulations are incredibly expensive to run. To perform optimization tasks—like finding the perfect drug dosing schedule—algorithms must simulate the model thousands of times, creating a massive computational bottleneck.
While fields like climate modeling and physics have standardized machine learning benchmarks (like WeatherBench or PDEBench) to train fast neural network "surrogates" that mimic these simulations at a fraction of the cost, systems biology has lacked an equivalent. Existing repositories share individual biological models but do not provide large, interconnected families of executable models paired with systematically generated simulation trajectories. PhysiBench addresses this fragmentation by offering a controlled, diverse, and reproducible testing ground for scientific machine learning and optimization algorithms.
Methods
The authors developed an automated pipeline to generate a vast, diverse suite of executable models and simulation data from seven established reference Boolean networks (covering cell-cycle control, cancer signaling, immune response, and more).
The pipeline operates in three main stages:
- Mutation-Based Generation: The reference models are stochastically mutated (e.g., swapping logic rules, replacing operators, adding nodes) to create structural variants. Crucially, every variant is bound to a standardized interface: exactly one input node (for external stimulation) and three output nodes (governing cell survival, proliferation, and death).
- Online Behavioral Filtering: To eliminate redundant models, candidates are simulated under 48 distinct protocols. Models whose population-level behavioral signatures correlate too highly (Pearson correlation > 0.85) with already accepted models are discarded.
- Offline Sensitivity Filtering: The remaining models are evaluated across 215 simulation contexts to ensure they actively respond to changes in input parameters, filtering out models that are executable but dynamically inert.
Finally, the authors selected 60 highly diverse models from this suite and ran 2,000 independent simulations per model using the hybrid agent-based framework PhysiBoSS/PhysiCell, varying the stimulation protocols and initial spatial configurations.
Datasets
PhysiBench introduces two primary linked components:
- Benchmark Model Suite: A collection of 612 executable intracellular Boolean regulatory network variants derived from 7 distinct biological families. They are provided in standard MaBoSS-compatible
.bndand.cfgformats. - Multiscale Trajectory Dataset: A 23 GB dataset containing 120,000 time-resolved simulations (2,000 runs for each of the 60 selected models). The data includes full cell-level spatial coordinates ($x, y, z$) and cell-cycle/death-state annotations across time, completely mapped to their respective stochastic seeds and input parameters.
Key Insights and Ideas
- Interface Standardization is Powerful: Enforcing a uniform 1-input, 3-output interface across entirely different biological systems allows researchers to test a single machine learning architecture or optimization algorithm across hundreds of distinct dynamical regimes without rewriting code.
- Synthetic Diversity Trumps Manual Curation: Manually building hybrid multiscale models is incredibly resource-intensive. This paper proves that combining stochastic network mutation with strict behavioral filtering can successfully generate structurally sound, highly diverse networks that maintain traceable lineages to real biological systems.
- Exceptionally Low Redundancy: The filtering pipeline was highly effective; the average correlation between the output-response profiles of the selected dataset models is just 0.06015, indicating highly independent and diverse dynamic behaviors.
Key Results
- The pipeline successfully whittled down over 200,000 test candidates to 612 sensitive, non-redundant models.
- Generating the 120,000-trajectory dataset required ~2,543 CPU-hours on Intel Xeon Gold processors, representing a substantial computational investment that researchers can now bypass.
- Technical Validation: The structural diversity of the suite was rigorously validated using three graph-based distance metrics (DeltaCon, Ipsen–Mikhailov distance, and Quantum Jensen–Shannon divergence), confirming that the mutation process yielded genuinely distinct regulatory structures rather than minor syntactic variations.
Limitations and Future Work
- Methodological, Not Clinical: The authors explicitly state that these models are synthetic constructs designed for algorithmic benchmarking. They are not independently calibrated against experimental data and should not be used to predict real-world clinical drug efficacy or toxicity.
- Boolean Simplification: The discrete, switch-like nature of Boolean logic cannot capture graded signal transduction or continuous dose-response relationships.
- Dimensionality Constraints: The multiscale simulations are currently restricted to a 2D spatial domain, meaning they omit 3D tissue architecture and complex extracellular matrix mechanics that influence real-world cell behavior. Future extensions could expand these benchmarks into full 3D environments.
Summary updated