DLGA-PDE: Discovery of PDEs with incomplete candidate library via combination of deep learning and genetic algorithm

Hao Xu, Haibin Chang, Dongxiao Zhang

TL;DR

This paper introduces DLGA-PDE, a novel framework that combines deep learning and genetic algorithms to discover partial differential equations (PDEs) from data. Its key innovation is the ability to find governing PDEs even when the set of possible terms (the "candidate library") is incomplete, a significant limitation of previous methods. This makes it more practical for real-world problems where the exact form of the governing equations is unknown.

What Problem Does This Paper Address?

Discovering the underlying mathematical equations that govern physical phenomena is a fundamental challenge in science and engineering. Traditionally, this involves expert knowledge and intuition to propose candidate equations. More recently, data-driven methods have emerged, aiming to automatically identify these equations from observed data. However, most existing data-driven approaches, particularly those using sparse regression or neural networks, rely on a complete library of potential terms that must contain the true governing equation. Constructing such a complete library is often difficult or impossible in practice, especially for complex systems. This paper addresses this critical gap by proposing a method that can discover PDEs even when the candidate library is incomplete.

Methods

The core of DLGA-PDE is a two-stage process:

  1. Deep Learning for Data Augmentation and Derivative Calculation: A deep neural network (specifically, a feed-forward fully-connected ANN) is trained on the available observational data. This trained network then serves to approximate the underlying physical system and, crucially, to accurately calculate derivatives of the observed variables. This is a significant advantage over traditional finite difference methods, which can be sensitive to noise.
  2. Genetic Algorithm for PDE Discovery: A genetic algorithm is employed to search for the governing PDE.
  3. Encoding: A novel encoding scheme is introduced to represent PDEs as "genomes." This scheme allows for flexibility in representing different temporal derivatives on the left-hand side of the equation and various combinations of spatial derivatives and powers of the dependent variable on the right-hand side.
  4. Incomplete Library Handling: The genetic algorithm's inherent mechanisms of crossover (combining parts of parent genomes) and mutation (randomly altering genes) allow it to explore a vast search space and generate novel combinations of terms. This is how DLGA-PDE overcomes the limitation of an incomplete candidate library; it can "discover" new terms through mutation.
  5. Fitness Function: A fitness function is defined to evaluate the quality of each candidate PDE. It combines the mean squared error (MSE) from fitting the PDE to the data with an L0-regularization term to encourage parsimony (fewer terms).

Datasets

The paper uses synthetic datasets generated from known PDEs to evaluate DLGA-PDE. The specific PDEs used are:

  • Korteweg-de Vries (KdV) equation
  • Wave equation
  • Burgers equation
  • Chaffee-Infante equation

The datasets are generated using various numerical methods (spectral, central difference, forward difference) and then subjected to different levels of noise and limited data scenarios to test the robustness of the proposed method. No new benchmarks are introduced; rather, standard PDEs are used for validation.

Key Insights and Ideas

  • Breaking the "Complete Library" Barrier: The most significant insight is that by combining the derivative-calculating power of deep learning with the exploratory search capabilities of genetic algorithms, it's possible to discover governing equations even when the initial set of potential terms is incomplete. This dramatically expands the applicability of data-driven PDE discovery.
  • Mutation as a Discovery Engine: The mutation operators in the genetic algorithm are not just for fine-tuning but are essential for generating novel terms (e.g., higher-order derivatives) that might be missing from the initial candidate set. This is a powerful mechanism for true equation discovery.
  • Flexibility in Left-Hand Side: The encoding scheme allows for flexible identification of the temporal derivative term (e.g., $u_t$, $u_{tt}$), which is often a fixed assumption in other methods.
  • Robustness to Noise and Limited Data: The combination of a trained neural network for derivative calculation and the genetic algorithm's search strategy makes the method surprisingly robust to noisy and limited data.

Key Results

  • Successful Discovery of Multiple PDEs: DLGA-PDE successfully discovered the correct forms of the KdV, wave, Burgers, and Chaffee-Infante equations, even when the candidate libraries were intentionally made incomplete.
  • Accurate Coefficient Estimation: The method accurately estimated the coefficients of the discovered PDEs.
  • Robustness Demonstrated: The method performed well with limited data (as low as 0.83% of total data for Chaffee-Infante) and with up to 20% noise in the data for the Burgers equation.
  • Overcoming Local Minima: The genetic algorithm's mutation process was shown to help the search escape local minima, as demonstrated in the Chaffee-Infante equation discovery, where it took many generations and a jump out of a local minimum to find the correct structure.
  • Comparison with DL-PDE: DLGA-PDE outperformed DL-PDE (a sparse regression-based method) on the Chaffee-Infante equation, which required a larger candidate library. DLGA-PDE's ability to generate terms via mutation made it more stable and successful in this complex case.

Limitations and Future Work

  • Hyperparameter Tuning: The L0-regularization hyperparameter is currently set by experience, and the authors suggest a need for a more sophisticated method for its adjustment.
  • Constant Coefficients: The current DLGA-PDE framework assumes constant coefficients in space and time. It cannot handle piecewise-constant or smoothly varying coefficients, which is identified as an area for future work.
  • Computational Cost: While the method converges quickly in many cases, the genetic algorithm can still be computationally intensive, especially for very complex problems or large populations/generations.

Summary updated

Abstract

Data-driven methods have recently been developed to discover underlying partial differential equations (PDEs) of physical problems. However, for these methods, a complete candidate library of potential terms in a PDE are usually required. To overcome this limitation, we propose a novel framework combining deep learning and genetic algorithm, called DLGA-PDE, for discovering PDEs. In the proposed framework, a deep neural network that is trained with available data of a physical problem is utilized to generate meta-data and calculate derivatives, and the genetic algorithm is then employed to discover the underlying PDE. Owing to the merits of the genetic algorithm, such as mutation and crossover, DLGA-PDE can work with an incomplete candidate library. The proposed DLGA-PDE is tested for discovery of the Korteweg-de Vries (KdV) equation, the Burgers equation, the wave equation, and the Chaffee-Infante equation, respectively, for proof-of-concept. Satisfactory results are obtained without the need for a complete candidate library, even in the presence of noisy and limited data.
BibTeX
@article{xu2020dlga,
  author = {Hao Xu and Haibin Chang and Dongxiao Zhang},
  title = {{DLGA-PDE: Discovery of PDEs with incomplete candidate library via combination of deep learning and genetic algorithm}},
  journal = {Journal of Computational Physics},
  year = {2020},
  doi = {10.1016/j.jcp.2020.109584},
}
Figure