Solving Fokker-Planck equations using deep KD-tree with a small amount of data

Yong Xu, Hao Zhang, Qi Liu, Xiaolong Wang, Yongge Li

Abstract

The Fokker-Planck (FP) equation deterministically describes the evolution of the probability density function (PDF), playing a significant role in stochastic dynamics. However, engineering constraints often limit available samples, hindering traditional solution methods. This study establishes the super-DL-FP framework to solve the steady-state FP equation with minimal data by combining deep learning with an innovative deep KD-tree method. A critical challenge in solving the steady-state FP equation is the normalization condition, which requires the PDF to integrate to unity. The deep KD-tree method estimates this integral by executing multiple KD-tree segmentations on random data within the integral region, identifying appropriate discrete integral points and volumes. Numerical experiments demonstrate that the super-DL-FP method achieves higher accuracy at a lower computational cost compared to center difference schemes, Chebyshev spectrum algorithms, and normalized flow approaches.

Introduction

The Fokker-Planck (FP) equation is a fundamental tool for modeling systems influenced by random noise, ranging from chemical kinetics to structural engineering. Despite its importance, solving the FP equation remains computationally intensive due to two primary obstacles:

  1. The Curse of Dimensionality: Grid-based numerical methods (e.g., finite element or finite difference) scale exponentially with the number of dimensions, making high-dimensional systems intractable.
  2. The Normalization Constraint: A valid PDF must integrate to exactly 1. In deep learning frameworks, enforcing this global constraint typically requires massive datasets to prevent the model from converging to unnormalized local solutions.

In practical engineering scenarios, data is often scarce. The super-DL-FP framework addresses these challenges by enabling accurate PDF estimation from small, random experimental datasets.

Methodology

The super-DL-FP framework integrates deep learning with a novel spatial partitioning technique.

Deep Learning for FP Equations (DL-FP)

The framework utilizes a neural network as a universal function approximator to represent the PDF. Unlike traditional grid-based methods, neural networks allow for analytical calculation of derivatives, facilitating the handling of off-grid points and complex geometries.

The Deep KD-Tree Innovation

The core contribution of this work is the deep KD-tree method, designed to estimate the normalization integral from sparse, non-uniform data. While a standard KD-tree partitions space into rectangular "boxes" based on data points, the authors employ an iterative "deep" version to determine the "volume" associated with each random sample.

The deep KD-tree partitioning process: (a) raw data, (b) initial KD-tree split, (c) adding geometric boundaries, and (d) final split after boundary integration.

The process follows a "peeling the fruit" analogy: the algorithm identifies and separates boundary points before re-partitioning the interior. This allows for effective numerical integration on non-uniform datasets. The final training loss function incorporates the FP equation residuals, boundary conditions, and the KD-tree-derived normalization integral.

Experimental Evaluation

The researchers validated the super-DL-FP method using synthetic experimental data generated via Stochastic Runge-Kutta (SRKII) simulations across several systems:

  • 2D Van der Pol System: Used to test the model's ability to capture complex PDF shapes, such as "crater" distributions.
  • 4D Coupled System: A test of higher-dimensional variable interaction and non-linear potentials.
  • 6D Stochastic System: A high-dimensional case designed to demonstrate scalability where grid-based methods fail.

Results and Discussion

The super-DL-FP method demonstrated significant advantages in data efficiency and accuracy:

  • High Accuracy with Sparse Data: In the 2D Van der Pol system, the model achieved 99.30% accuracy with 1,000 points and maintained 84.21% accuracy with only 100 points.
  • Superior Performance: Compared to traditional methods like Central Difference and Sylvester Flow at 200 calculation points, super-DL-FP reduced error magnitudes from $10^{-1}$ to $10^{-2}$.
  • Scalability: The framework successfully solved the 6D system using 200,000 points—a task that would require billions of points under standard discretization schemes.

Key insights include the discovery that using the geometric midpoint of KD-tree partitions as integral points significantly enhances stability. Furthermore, because the solution is a neural network, it is inherently analytical, allowing for instantaneous probability queries at any coordinate.

Conclusion

The super-DL-FP framework provides a robust solution for solving Fokker-Planck equations in data-limited environments. By bridging discrete random samples and continuous calculus through the deep KD-tree, the model bypasses the limitations of traditional grid-based methods. Future work will focus on providing a rigorous mathematical proof of the deep KD-tree's convergence and exploring more sophisticated integral schemes, such as quantile integrals, to further refine accuracy across diverse physical and engineering applications.

Summary updated

Abstract

<title>Abstract</title> The Fokker-Planck (FP) equation can deterministically describe the evolution of the probability density function, which plays an extremely significant role in the fields of stochastic dynamics. Unfortunately, the limited samples that arise from the consideration of engineering practice are inevitable, which restricts the solving of the FP equation. Accordingly, in the present study, a super-DL-FP framework is established to solve the steady-state FP equation with a small amount of data, through combining the deep KD-tree and the DLFP approach proposed in [Chaos 30, 013133 (2020)]. It should be emphasized that the normalization condition is of great importance and have to be considered in solving the steady-state FP equation. An appropriate integral estimation for the normalization condition under non-uniform meshing can effectively improve the precision of the solution, but it is still a challenging problem, especially for the case of small data. Thus, the so-called deep KD-tree method is innovatively proposed to estimate the normalized integral with a small random dataset. The main target is to obtain the appropriate discrete integral points and corresponding integral volumes by executing multiple KD-tree segmentation based on random data on the integral region. Several numerical experiments and comparisons are implemented to illustrate the superior performance of the super-DL-FP method. The obtained results indicate that the proposed algorithm can accomplish higher accuracy in the sense of lower cost than the well-known algorithms like center difference scheme, Chebyshev spectrum algorithm, and normalized flow approach.
BibTeX
@inproceedings{xu2022solving,
  author = {Yong Xu and Hao Zhang and Qi Liu and Xiaolong Wang and Yongge Li},
  title = {{Solving Fokker-Planck equations using deep KD-tree with a small amount of data}},
  booktitle = {Nonlinear},
  year = {2022},
  doi = {10.1007/s11071-022-07361-2},
}
Figure