Automatically Learning Hybrid Digital Twins of Dynamical Systems

Samuel Holt, Tennison Liu, Mihaela van der Schaar

TL;DR

Modeling complex dynamical systems usually requires a painful trade-off: use rigid mechanistic equations that lack flexibility, or deploy massive neural networks that fail when data is scarce. This paper introduces HDTwinGen, an evolutionary framework that leverages Large Language Models (LLMs) to automatically code, evaluate, and optimize Hybrid Digital Twins (HDTwins). The resulting models are highly sample-efficient, generalize remarkably well out-of-distribution, and can adapt to unseen system interventions by modifying a single parameter.

What Problem Does This Paper Address?

Digital Twins (DTs) are computational models designed to simulate real-world dynamical systems—such as the spread of an infectious disease or the growth of a tumor. To be truly useful, a Digital Twin must satisfy three core criteria:

  1. Out-of-distribution (OOD) generalization: It must accurately simulate completely unseen scenarios or interventions at inference time.
  2. Sample efficiency: It must learn robustly from minimal empirical data.
  3. Evolvability: It must easily adapt to changes in the underlying system dynamics (e.g., a virus mutating or a patient developing drug resistance).

Traditional approaches fall into two camps: mechanistic models (closed-form equations grounded in physics/biology) and neural models (purely data-driven deep learning). Mechanistic models generalize well but fail when our scientific understanding of a system is incomplete. Neural models are highly expressive but require massive datasets, struggle to generalize OOD, and are difficult to update due to their monolithic nature.

Hybrid Digital Twins (HDTwins) combine both, typically using an additive structure ($f = f_{mech} + f_{neural}$) where a neural network models the residuals that the physics equations miss. However, designing the right hybrid architecture has traditionally required slow, manual, expert human engineering. Automatically searching the massive combinatorial space of equations and neural network structures has remained computationally intractable—until now.

Methods

The authors propose HDTwinGen, an automated evolutionary framework that splits the problem into a bilevel optimization task: model specification (structural design) and model parameterization (parameter fitting).

Instead of relying on rigid genetic algorithms, HDTwinGen uses a multi-agent LLM architecture to navigate the structural search space:

  1. The Modeling Agent (LLM): Given a natural language description of the system and a basic code skeleton, this agent proposes a candidate model specification written in Python/PyTorch.
  2. Offline Parameter Optimization: Once the code structure is generated, standard gradient-based optimization tools (like the Adam optimizer) take over to fit the continuous parameters (weights, coefficients) directly to the training data.
  3. The Evaluation Agent (LLM): This agent acts as a critic. It reviews the top-performing models, analyzes component-wise validation losses, and provides targeted, natural language feedback (e.g., "Introduce a non-linear term like a Michaelis-Menten equation to model chemo saturation").

This loop repeats iteratively, allowing the system to evolve increasingly accurate and interpretable hybrid models.

Figure 1: HDTwinGen: evolutionary framework. The process begins with user-provided modeling context Scontext and D = {Dtrain, Dval}. 1) In iteration g, the modeling agent generates model specification as a Python program fθ,ω(θ). 2) Parameters are optimized using the offline optimization tool to yield fθ,ω∗(θ). 3) The HDTwin is evaluated based on model loss υ and component-wise loss δ. Subsequently, the model pool P(g) is updated with top-K models. 4) The evaluation agent provides targeted feedback for model improvement H(g) by analyzing models in P(g) using performance metrics requirements outlined in Scontext. This iterative loop repeats for G iterations.

Datasets

The framework was evaluated across six complex system benchmarks, spanning biomedical, ecological, and epidemiological domains:

  • Lung Cancer (3 variants): Simulating tumor volume dynamics under no treatment, chemotherapy only, and combined chemo- and radiotherapy based on Pharmacokinetic-Pharmacodynamic (PKPD) models.
  • COVID-19: An epidemic agent-based simulator (COVASIM) modeling disease spread and non-pharmaceutical interventions.
  • Plankton Microcosm: A real-world ecological dataset tracking a three-species prey-predator system (algae, flagellate, rotifer).
  • Hare-Lynx: A classic real-world dataset capturing predator-prey population oscillations.

To ensure the LLM wasn't simply memorizing known equations, the authors also tested HDTwinGen on procedurally generated synthetic benchmarks containing non-biological random modifications (like arbitrary trigonometric and division operators).

Key Insights and Ideas

  • Code as the Universal Interface: By representing model structures as executable Python/PyTorch code, the framework allows LLMs to seamlessly combine symbolic mathematics with deep neural network layers.
  • Natural Language as a Dense Feedback Signal: Unlike traditional reinforcement learning or genetic algorithms that rely on sparse scalar rewards, HDTwinGen uses rich, qualitative critiques from the Evaluation Agent. This mimics the way human scientists collaborate and refine hypotheses.
  • True Modular Evolvability: Because the generated models maintain a symbolic, modular structure, they can adapt to completely unobserved interventions (such as a sudden pandemic lockdown) by adjusting a single parameter. Monolithic neural networks cannot do this without a brand-new training dataset.

Key Results

HDTwinGen consistently outperformed state-of-the-art neural sequence models (DyNODE, RNNs, Transformers), pure equation discovery methods (SINDy, Genetic Programming), and existing hybrid frameworks (APHYNITY).

Table 1: Benchmark method performance. Reporting the test prediction MSE (TMSE) of the produced system models on held-out test datasets across all benchmark datasets. HDTwinGen achieves the lowest test prediction error. The results are averaged over ten random seeds, with ± indicating 95% confidence intervals.

Out-of-Distribution Generalization

When tested on state-action distributions completely outside the training range, HDTwinGen maintained high accuracy, while purely data-driven models degraded drastically. For instance, on the Lung Cancer OOD benchmark, HDTwinGen achieved a Test MSE of 0.0846, compared to DyNODE's 1.75 and the Transformer's 11,900.

Table 2: Out of distribution shifts. On a varia- tion of the Lung Cancer (with Chemo. & Radio.), HDTwinGen is more robust to OOD shifts in un- seen state-action distributions.

Sample Efficiency

HDTwinGen proved exceptionally robust in data-scarce environments, achieving lower prediction errors when trained on just a handful of trajectories than competing models could achieve with hundreds.

Figure 2: Sample efficiency. Analyzing perfor- mance as a function of the number of training trajectories in the Lung Cancer (with Chemo. & Radio.) dataset. We observe that HDTwinGen achieves the lowest test prediction error, even in the very challenging low data regime. This high- lights the role of priors embedded in HDTwin in sample-efficient generalization.

Speed and Efficiency

In a comparison against human software engineers (via Mechanical Turk), HDTwinGen completed the 15-generation design process in an average of 45.56 minutes, whereas human experts took 148.1 minutes to produce models with significantly higher error rates.

Limitations and Future Work

While powerful, HDTwinGen has a few notable limitations:

  • Prompt Dependency: It relies on a human expert to provide a comprehensive initial prompt outlining the system variables and modeling constraints.
  • LLM Bottlenecks: The framework's success is inherently bounded by the domain knowledge and code-generation capabilities of the underlying LLM (e.g., GPT-4 performed significantly better than GPT-3.5).
  • Continuous-Time Restriction: The current implementation focuses exclusively on continuous-time systems governed by differential equations, leaving discrete-time or hybrid-time systems for future exploration.

The authors suggest that future iterations could benefit from direct human-in-the-loop feedback during the evolutionary process and integration with external symbolic algebra tools.

Summary updated

Abstract

Digital Twins (DTs) are computational models that simulate the states and temporal dynamics of real-world systems, playing a crucial role in prediction, understanding, and decision-making across diverse domains. However, existing approaches to DTs often struggle to generalize to unseen conditions in data-scarce settings, a crucial requirement for such models. To address these limitations, our work begins by establishing the essential desiderata for effective DTs. Hybrid Digital Twins ($\textbf{HDTwins}$) represent a promising approach to address these requirements, modeling systems using a composition of both mechanistic and neural components. This hybrid architecture simultaneously leverages (partial) domain knowledge and neural network expressiveness to enhance generalization, with its modular design facilitating improved evolvability. While existing hybrid models rely on expert-specified architectures with only parameters optimized on data, $\textit{automatically}$ specifying and optimizing HDTwins remains intractable due to the complex search space and the need for flexible integration of domain priors. To overcome this complexity, we propose an evolutionary algorithm ($\textbf{HDTwinGen}$) that employs Large Language Models (LLMs) to autonomously propose, evaluate, and optimize HDTwins. Specifically, LLMs iteratively generate novel model specifications, while offline tools are employed to optimize emitted parameters. Correspondingly, proposed models are evaluated and evolved based on targeted feedback, enabling the discovery of increasingly effective hybrid models. Our empirical results reveal that HDTwinGen produces generalizable, sample-efficient, and evolvable models, significantly advancing DTs' efficacy in real-world applications.
BibTeX
@inproceedings{holt2024automatically,
  author = {Samuel Holt and Tennison Liu and Mihaela van der Schaar},
  title = {{Automatically Learning Hybrid Digital Twins of Dynamical Systems}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=SOsiObSdU2},
}
Figure