Abstract
Learning dynamical systems from observed trajectories is a fundamental problem in data-driven science and engineering. While many existing works focus on improving model architectures or training methods, less attention has been directed at how to effectively sample training data to give rise to accurate models. In particular, one of the most basic problems is to select the length of sampled trajectories that balances computational overhead due to sampling and the quality of learned models. This paper deals with the task of improving sampling efficiency for learning dynamics. We first formulate proper target risks to evaluate the model performance of learning in the dynamical setting. This allows us to connect generalization to matching empirical measures with specific target measures. In line with this observation, we propose a class of adaptive algorithms to find effective sampling strategies that control the length of sampled trajectories. Through numerical experiments, we show the adaptive algorithms can achieve more accurate results given a sampling budget compared to baseline sampling methods.
Introduction
In the field of data-driven modeling, researchers often face a "chicken-and-egg" problem: obtaining an accurate model requires high-quality data, but identifying which data is "high quality" often requires an understanding of the underlying dynamics. While much effort has been spent on developing sophisticated architectures like Physics-Informed Neural Networks, the sampling strategy—specifically the length of training trajectories—is frequently overlooked.
Sampling is often the most expensive part of the pipeline, especially in high-fidelity simulations or real-world experiments. If trajectories are too short, the model may miss critical long-term behaviors; if they are too long, computational resources are wasted on redundant data. This research introduces a principled framework for adaptive sampling that automatically determines optimal trajectory lengths to maximize data efficiency.
Methodology: Generalization as Measure Matching
The core insight of this work is that "good" sampling can be framed as a measure-matching problem. The authors provide a formal proof that the target risk of a model is bounded by the discrepancy between the empirical measure (the training data) and a target measure (the behavior to be learned).
Key Techniques
- 1-Wasserstein Distance ($W_1$): This metric is used to quantify the distance between the training data distribution and the target distribution. As shown in the figure below, the $W_1$ distance serves as a reliable proxy for the actual target risk.
- Adaptive Stopping Criteria: Rather than using a fixed trajectory length $T$, the proposed algorithms monitor the rate of change in the $W_1$ distance. By adding a cost term $\phi$, the sampler can identify the "sweet spot" where the benefit of additional data no longer justifies the computational expense.
- Trajectory-wise Adaptivity: The algorithms can detect redundancy in real-time. If a new trajectory enters a region of the state space already well-covered by existing data, it is terminated early to reallocate resources to unexplored regions.
Experimental Evaluation
The framework was evaluated across several classic and high-dimensional dynamical systems, including the chaotic Lorenz 63 system, the Van der Pol Oscillator, and a 50-dimensional Ginzburg-Landau Equation derived from a discretized PDE.
Key Results
- Efficiency Gains: In the high-dimensional Ginzburg-Landau system, the adaptive sampler achieved the same accuracy as baseline methods while requiring 26.3% fewer state samples.
- Superior Accuracy: Given a fixed sampling budget, the adaptive algorithms consistently produced lower target risks compared to passive, fixed-length sampling.
- Model Agnostic: The methods proved effective across various learning models, including standard Fully Connected Neural Networks, Random Feature Models, and Sparse Identification of Nonlinear Dynamics (SINDy).
Discussion and Future Work
While the results demonstrate significant efficiency gains, calculating the 1-Wasserstein distance (typically via the Sinkhorn algorithm) introduces its own computational overhead. Although this is often negligible compared to expensive physical simulations, it remains a factor for very large datasets.
Future research will focus on adaptively choosing the initial conditions of trajectories—not just their length—and establishing more rigorous theoretical guarantees for the convergence of these adaptive sampling loops.
Summary updated