TL;DR
This paper introduces a novel deep-learning framework that discovers partial differential equations (PDEs) in their integral form, overcoming limitations of existing methods in handling sparse, noisy data, and complex PDEs with high-order derivatives or heterogeneous parameters. By combining deep neural networks for data augmentation and derivative calculation with a genetic algorithm for structure discovery, the method achieves robust and accurate PDE identification.
What Problem Does This Paper Address?
Discovering the underlying partial differential equations (PDEs) that govern physical phenomena from observational data is a crucial task in scientific research. While significant progress has been made, existing data-driven methods struggle with several challenges:
- High-order derivatives: Accurately calculating high-order derivatives from noisy and sparse data is notoriously difficult, leading to poor performance in discovering PDEs with such terms.
- Heterogeneous parameters: Identifying PDEs where parameters vary spatially (e.g., in heterogeneous media) is complex, as these parameters are often embedded within the differential operators, making them hard to isolate.
- Sparse and noisy data: Many real-world datasets are incomplete and contain noise, which can severely degrade the accuracy of PDE discovery methods.
This paper aims to address these limitations by proposing a unified framework that leverages the integral form of PDEs, which is inherently more robust to noise and can simplify the identification of complex terms.
Methods
The proposed framework combines deep learning with an integral form approach and a genetic algorithm:
- Deep Neural Network (DNN) for Data Augmentation: A DNN is trained on the sparse observation data to approximate the underlying system. This trained network then generates a dense set of "meta-data" and accurately calculates derivatives via automatic differentiation. This meta-data is crucial for overcoming data sparsity.
- Integral Form Transformation: The core innovation is transforming the PDEs into their integral (or weak) form. This process naturally reduces the order of derivatives required for calculation, making the method more robust to noise and simplifying the identification of high-order terms. A unified integral form is derived that accommodates both constant and heterogeneous parameters.
- Genetic Algorithm for Structure Discovery: A genetic algorithm is employed to search for the most parsimonious and accurate PDE structure in its integral form. The algorithm works by:
- Digitization: Representing PDE terms as "genes" and combining them into "genomes" representing candidate PDEs.
- Fitness Calculation: Evaluating the quality of each candidate PDE by calculating the mean squared error (MSE) between the learned equation and the meta-data, penalized by genome length.
- Evolution: Using crossover and mutation operations to evolve a population of candidate PDEs towards better solutions.
- Stepwise Method for Heterogeneous Parameters: For PDEs with heterogeneous parameters, a stepwise approach is used. First, the PDE structure is discovered in local spatial windows. Then, global meta-data and the discovered structure are used to calculate the values of the heterogeneous parameters. A coefficient of variation criterion is used to distinguish between constant and variable parameters.
Datasets
The paper utilizes synthetically generated datasets for various PDEs to evaluate the proposed method. These include:
- Korteweg-de Vries (KdV) equation: Used to test discovery with high-order derivatives and sparse/noisy data.
- Kuramoto-Sivashinsky (KS) equation: Another test case for high-order derivatives and noise robustness.
- Heterogeneous parametric convection-diffusion equation: Used to evaluate the discovery of PDEs with spatially varying parameters.
- Heterogeneous parametric wave equation: Another test case for heterogeneous parameters, including varying amplitudes.
- Heterogeneous parametric Boussinesq equation: Used to test the method's ability to handle complex PDEs with strong nonlinearity.
The datasets are generated using numerical methods (spectral, finite difference) and are described in detail in Appendix A.
Key Insights and Ideas
- Integral Form is Key: Transforming PDEs into their integral form is a powerful strategy. It inherently smooths out noise and reduces the order of derivatives needed, leading to more accurate and robust discovery, especially for high-order PDEs.
- DNNs as Data Augmentors: Deep neural networks are not just for solving but also for generating high-quality, dense meta-data from sparse observations, which is critical for subsequent analysis.
- Unified Integral Form: The development of a single integral form that handles both constant and heterogeneous parameters simplifies the discovery process and broadens the applicability of the method.
- Genetic Algorithm for Structure Search: The genetic algorithm provides a flexible and effective way to explore the vast space of possible PDE structures, especially when the candidate library is incomplete or unknown.
- Stepwise Heterogeneous Parameter Identification: Separating structure discovery from parameter estimation, especially for heterogeneous parameters, allows for a more manageable and accurate approach.
Key Results
The proposed method demonstrates strong performance across various challenging scenarios:
- Robustness to Noise: The integral form approach is significantly more robust to noise than differential form methods. For instance, it can discover the KdV equation with up to 25% noise, while differential form methods struggle beyond 15%.
- Handling Sparse Data: The method can accurately discover PDEs even with very limited data. The KdV equation was identified using only 500 data points (0.486% of total).
- High-Order Derivatives: The method successfully identifies PDEs with high-order derivatives, such as the KdV (3rd derivative) and KS (4th derivative) equations, with high accuracy.
- Heterogeneous Parameters: The framework accurately identifies both the structure and the values of heterogeneous parameters in PDEs, even when they are random fields or have varying amplitudes. For example, it correctly identified constant parameters with high confidence and accurately estimated varying parameters.
- Accuracy: In most experiments, the learned equations closely match the true PDEs, with errors typically in the low single digits for clean data and manageable levels for noisy or sparse data.
Limitations and Future Work
The authors acknowledge several limitations and suggest future research directions:
- Complex Dynamics: For some governing equations with highly complex dynamics, their integral forms might not be representable by simple combinations of spatial derivatives, limiting the method's applicability.
- Computational Cost: The integration process over each interval can be time-consuming, making the method computationally intensive.
- Hyper-parameter Tuning: The choice of hyper-parameters (e.g., integral interval length, genetic algorithm parameters) can affect the discovered PDE and requires careful tuning, often through experimental comparison and posterior error analysis.
Future work could focus on developing more efficient integration techniques, exploring alternative optimization algorithms, and extending the method to handle PDEs where integral forms are not easily expressible.
Summary updated