Learning Generalized Diffusions Using an Energetic Variational Approach

Yubin Lu, Xiaofan Li, Chun Liu, Qi Tang, Yiwei Wang

TL;DR

This paper introduces a novel data-driven framework for learning the governing laws of dissipative physical systems, specifically generalized diffusions. By leveraging the energy-dissipation law and an energetic variational approach, the method can infer system dynamics from either continuous density data or discrete particle trajectories, offering robustness to noise and potential for high-dimensional problems.

What Problem Does This Paper Address?

Many physical systems, like fluid dynamics and plasma physics, exhibit dissipative behavior due to factors like viscosity or collisions. Understanding and modeling these systems often involves discovering their underlying physical laws, typically expressed as nonlinear partial differential equations (PDEs). Traditional data-driven methods for this task, such as Physics-Informed Neural Networks (PINNs) or Sparse Identification of Nonlinear Dynamics (SINDy), often rely on the explicit form of the governing equations. This can be challenging for complex systems or when dealing with noisy data. Furthermore, ensuring that learned models preserve fundamental physical principles, like thermodynamic consistency, is crucial but not always guaranteed. This paper addresses the need for a robust, generalizable, and physically consistent approach to learning these dissipative dynamics directly from data, without prior knowledge of the governing equations.

Methods

The core of this work lies in the Energetic Variational Approach (EnVarA), which models physical systems using an energy-dissipation law. The authors propose a learning framework that constructs a loss function directly from this energy-dissipation law, rather than from the governing PDEs themselves. This approach has several advantages:

  • Energy-Dissipation Law as the Basis: Instead of solving PDEs, the framework uses the integral form of the energy-dissipation law. This is less sensitive to noise and requires weaker regularity conditions on the data.
  • Two Data Modalities: The framework accommodates two types of data:
  • Density-based Method: Uses probability density functions at three time instances to approximate the change in free energy and the dissipation rate.
  • Particle-to-Density Method: Leverages particle trajectories from simulations. Particle data is first used to estimate the probability density function (e.g., via Kernel Density Estimation), which is then used in the density-based loss function. This is particularly useful for high-dimensional problems where direct density estimation is intractable.
  • Learning Potential and Noise Intensity: The framework is designed to learn either the potential function ($\psi$) or the noise intensity ($\sigma^2$) of the generalized diffusion, assuming the other is known.
  • Neural Network Approximation: The unknown potential function is approximated by a neural network.

Datasets

The paper uses simulated data generated from solving the Fokker-Planck equation (2.4) or simulating the stochastic differential equation (2.3). These simulations provide either:

  • Probability density functions at specific time instances.
  • Particle trajectories at specific time instances.

The authors do not introduce new benchmark datasets but rather use synthetic data generated under controlled conditions to test their method's performance and robustness.

Key Insights and Ideas

  • Learning from Energy-Dissipation is Robust: Deriving the loss function directly from the energy-dissipation law, rather than the governing PDE, makes the method more robust to noisy observations. This is because the integral form of the loss function is less sensitive to local errors.
  • Particle Data for High Dimensions: The particle-to-density approach is a crucial innovation for tackling high-dimensional systems where obtaining accurate density data is difficult. It allows learning from readily available particle trajectories.
  • Steady-State Data for Ill-Posed Problems: Learning the potential function from steady-state data can help overcome the ill-posedness inherent in inverse problems, leading to more accurate reconstructions.
  • Noise Intensity Learning is More Stable: The paper suggests that learning the noise intensity ($\sigma^2$) is generally more robust than learning the potential function ($\psi$). This is attributed to $\sigma^2$ appearing in a simpler term within the loss function, making it more convex with respect to $\sigma^2$.

Key Results

  • Accurate Potential and Noise Learning: The proposed methods successfully learned both the potential function and the noise intensity in 1D and 2D generalized diffusion examples.
  • Improved Accuracy with More Data: Performance generally improved with an increased number of data groups (M) for both density-based and particle-to-density methods.
  • Density-Based Outperforms Particle-Based: In comparable scenarios, the density-based method generally yielded more accurate results than the particle-to-density method, likely due to the additional approximation error introduced by density estimation from particles. However, the particle-to-density method still provided reasonable results and is more practical for high dimensions.
  • Robustness to Noise: The density-based method demonstrated good robustness to observational noise, with learned potentials remaining reasonably accurate even with corrupted data.
  • 2D Example Success: The framework was successfully extended to a 2D system, demonstrating its scalability.

Limitations and Future Work

  • High-Dimensional Density Estimation: A significant challenge remains in efficiently and accurately estimating density functions in high-dimensional spaces, which is a prerequisite for the particle-to-density method. The authors suggest exploring alternative methods like normalizing flows for density estimation.
  • Combining Approaches: The paper suggests that combining weak-form loss functions with methods that use automatic differentiation might yield even more effective and robust learning frameworks.
  • Generalization to Other Systems: While the framework is designed to be generalizable, further work is needed to explore its application to more complex physical systems beyond generalized diffusions.
  • Data Availability: The paper notes that in many high-dimensional problems, density data is not readily available, necessitating its approximation from particle data. Future work could focus on developing loss functions that directly operate on particle data without explicit density estimation.

Summary updated

BibTeX
@misc{lu2025learning,
  author = {Yubin Lu and Xiaofan Li and Chun Liu and Qi Tang and Yiwei Wang},
  title = {{Learning Generalized Diffusions Using an Energetic Variational Approach}},
  year = {2025},
  doi = {10.2139/ssrn.5271190},
  howpublished = {SSRN},
}
Figure