Learning Dynamical Systems across Environments

Yuan Yin, Ibrahim Ayed, Emmanuel de Bézenac, Patrick GallinariAAAI 2021 · OpenReview

Abstract

Learning physical laws from data collected across varying environments is a fundamental challenge in scientific machine learning. Standard approaches often fail by either over-averaging across disparate data sources or overfitting to individual environments. We present LEADS (LEarning Across Dynamical Systems), a framework that decomposes dynamics into a shared universal component and an environment-specific residual. By enforcing a "smallness" constraint on the residual, LEADS captures invariant physical principles while remaining flexible enough to adapt to local variations. Our results on ODE and PDE benchmarks demonstrate significant improvements in accuracy and adaptation speed compared to state-of-the-art baselines.

Introduction

In many real-world physical systems, data is not independent and identically distributed (i.i.d.). Variations in spatial location, experimental setup, or individual biological characteristics introduce environmental shifts. For instance, the Coriolis force varies with latitude, and predator-prey dynamics depend on local resource availability.

Traditional modeling strategies typically fall into two traps:

  1. The "Average" Model: Treating all environments as a single dataset, which results in a biased model that fails to capture specific local nuances.
  2. The "Siloed" Model: Training independent models for each environment, which ignores shared underlying physics and requires large datasets for every new scenario.

LEADS provides a middle ground, leveraging shared knowledge to improve generalization and enable rapid adaptation to novel environments.

Methodology

The core of the LEADS framework is the decomposition of the governing differential equation for an environment $e$: $$\frac{dX_t}{dt} = g(X_t) + h_e(X_t)$$ where $g(X_t)$ represents the dynamics common to all environments, and $h_e(X_t)$ represents the environment-specific residual.

The "Smallness" Constraint

To ensure that $g$ captures the maximum amount of shared information, the authors introduce a constrained optimization problem. They minimize the norm of the residual ($\Vert h_e \Vert^2$) while satisfying the differential equation. This forces the shared component $g$ to explain as much of the data as possible, leaving $h_e$ to handle only what is truly unique. The training process utilizes a Lagrangian formulation integrated with Neural ODE/PDE solvers and employs scheduled sampling to maintain stability over long time horizons.

Experimental Results

The framework was evaluated on two classic synthetic benchmarks: the Lotka-Volterra (ODE) predator-prey model and the Gray-Scott (PDE) reaction-diffusion system.

Accuracy and Generalization

On the Lotka-Volterra task across 10 environments, LEADS achieved an 85% reduction in test Mean Squared Error (MSE) compared to environment-dependent baselines. Qualitatively, LEADS trajectories remain stable and accurate over time, whereas baseline models often accumulate errors and drift.

Comparison of trajectory predictions between a baseline model (a) and the LEADS framework (b). LEADS shows significantly better alignment with ground truth across multiple environments.

Rapid Adaptation

A key strength of LEADS is its ability to adapt to new environments. By using the learned shared component $g$ as a powerful prior, the model can reach a lower error in just 250 iterations in a new environment than a model trained from scratch can achieve in 10,000 iterations.

Discussion and Limitations

While LEADS demonstrates robust performance, it currently assumes an additive relationship between shared and specific dynamics ($g + h_e$). Future work may explore multiplicative or more complex couplings. Additionally, scaling the framework to high-dimensional systems like 3D turbulence remains a computational challenge for Neural ODE-based methods due to the cost of backpropagating through solvers.

Conclusion

LEADS successfully separates universal physical laws from local parameters through a regularized decomposition. This approach not only improves predictive accuracy but also provides a powerful mechanism for transfer learning in physical systems, moving us closer to models that can truly generalize across the diverse conditions of the real world.

Summary updated

BibTeX
@inproceedings{yin2021learning,
  author = {Yuan Yin and Ibrahim Ayed and Emmanuel de Bézenac and Patrick Gallinari},
  title = {{Learning Dynamical Systems across Environments}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=QRpfJPa7s7},
}
Figure