Generalizing to New Physical Systems via Context-Informed Dynamics Model

Matthieu Kirchmeyer, Yuan Yin, Jérémie Donà, Nicolas Baskiotis, Alain Rakotomamonjy, Patrick Gallinari

Abstract

Data-driven approaches to modeling physical systems fail to generalize to unseen systems that share the same general dynamics with the learning domain, but correspond to different physical contexts. We propose a new framework for this key problem, context-informed dynamics adaptation (CoDA), which takes into account the distributional shift across systems for fast and efficient adaptation to new dynamics. CoDA leverages multiple environments, each associated to a different dynamic, and learns to condition the dynamics model on contextual parameters, specific to each environment. The conditioning is performed via a hypernetwork, learned jointly with a context vector from observed data. The proposed formulation constrains the search hypothesis space to foster fast adaptation and better generalization across environments. We theoretically motivate our approach and show state-of-the-art generalization results on a set of nonlinear dynamics, representative of a variety of application domains. We also show, on these systems, that new system parameters can be inferred from context vectors with minimal supervision.

Introduction

Machine learning is increasingly used to emulate complex dynamical systems, offering faster alternatives to traditional numerical solvers for fluid dynamics, climate modeling, and biology. However, standard machine learning models rely on the assumption that training and testing data are independent and identically distributed (i.e., the i.i.d. assumption).

In the real world, physical systems are highly context-dependent. A model trained to predict fluid flow with a specific viscosity will fail catastrophically if tested on a fluid with a different viscosity, even though the underlying physical laws (such as the Navier-Stokes equations) remain the same. While meta-learning techniques like MAML exist to help models adapt to new tasks, they often struggle with complex differential equations, requiring heavy bi-level optimization and frequently overfitting when only a tiny amount of adaptation data (few-shot learning) is available.

The CoDA Framework

To address these challenges, we introduce Context-Informed Dynamics Adaptation (CoDA). Instead of trying to learn a single, domain-invariant model, CoDA learns a base dynamics model that can be dynamically shifted to fit new environments.

Architecture of the CoDA framework

The framework operates through three primary mechanisms:

  1. Hypernetwork-Based Conditioning: CoDA utilizes a base set of neural network weights. For any specific environment, a separate, low-dimensional "context vector" is learned. A linear hypernetwork decodes this context vector into a high-dimensional weight shift, which is then added to the base weights.
  2. Low-Rank Adaptation: Because the context vector is very small (e.g., 2 dimensions), the adaptation occurs in a low-rank subspace. This acts as a bottleneck, preventing the model from overfitting when adapting to a new environment based on limited data.
  3. Locality Constraint: The loss function includes a penalty that forces the environment-specific weights to stay close to the shared base weights.

During training, the model learns the base weights, the hypernetwork, and the context vectors for the training environments. During testing (adaptation), the network weights are frozen, and the model uses standard gradient descent to find the optimal context vector for the new, unseen environment.

Experimental Evaluation

We evaluated CoDA on four simulated dynamical systems covering both Ordinary Differential Equations (ODEs) and Partial Differential Equations (PDEs):

  • Lotka-Volterra (LV): Predator-prey population dynamics.
  • Glycolytic-Oscillator (GO): Highly non-linear yeast glycolysis.
  • Gray-Scott (GS): Reaction-diffusion patterns.
  • Navier-Stokes (NS): Incompressible fluid flows.

CoDA significantly outperformed multi-task and meta-learning baselines (such as MAML, ANIL, and LEADS) on both in-domain and out-of-domain adaptation tasks. Notably, CoDA successfully adapted to entirely new physical environments using only one trajectory (one-shot adaptation). While baselines severely overfit in this regime, CoDA maintained low error rates.

Insights: Low-Dimensionality and Parameter Discovery

A key insight of this work is that physics variations are inherently low-dimensional. The gradient directions required to adapt a model across different physical environments span a very low-dimensional subspace. For example, updating a 2D context vector is sufficient to account for changes in fluid viscosity.

Impact of context vector dimension on parameter estimation error

Furthermore, we discovered that the learned "context vectors" exhibit a linear bijection with the true, hidden physical parameters of the system. This allows CoDA to accurately measure unknown physical properties just by observing system movement. When guessing hidden parameters, CoDA achieved incredibly low Mean Absolute Percentage Errors (MAPE): 0.59% on Lotka-Volterra and 0.30% on Navier-Stokes.

Conclusion and Future Work

CoDA demonstrates that enforcing locality and low-rank constraints can lead to reliable, convex optimization for adaptation in physical systems. While the framework performs exceptionally well within the convex hull of training environments, its performance naturally degrades during extreme extrapolation. Future work will explore CoDA's application to partially observed systems and reinforcement learning in shifting environments, potentially leading to faster and more robust neural-network-based physics simulators.

Summary updated

Abstract

Data-driven approaches to modeling physical systems fail to generalize to unseen systems that share the same general dynamics with the learning domain, but correspond to different physical contexts. We propose a new framework for this key problem, context-informed dynamics adaptation (CoDA), which takes into account the distributional shift across systems for fast and efficient adaptation to new dynamics. CoDA leverages multiple environments, each associated to a different dynamic, and learns to condition the dynamics model on contextual parameters, specific to each environment. The conditioning is performed via a hypernetwork, learned jointly with a context vector from observed data. The proposed formulation constrains the search hypothesis space to foster fast adaptation and better generalization across environments. We theoretically motivate our approach and show state-of-the-art generalization results on a set of nonlinear dynamics, representative of a variety of application domains. We also show, on these systems, that new system parameters can be inferred from context vectors with minimal supervision. Code is available at https://github.com/yuan-yin/CoDA .
BibTeX
@inproceedings{kirchmeyer2022generalizing,
  author = {Matthieu Kirchmeyer and Yuan Yin and Jérémie Donà and Nicolas Baskiotis and Alain Rakotomamonjy and Patrick Gallinari},
  title = {{Generalizing to New Physical Systems via Context-Informed Dynamics Model}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=lkIKSkgNiMV},
}
Figure