Solving Fokker–Planck equations using deep KD-tree with a small amount of data

Hao Zhang, Yong Xu, Qi Liu, Xiaolong Wang, Yongge Li

TL;DR

Researchers have developed a new framework called super-DL-FP that solves Fokker-Planck equations—the mathematical backbone of stochastic dynamics—using significantly less data than traditional methods. By combining deep learning with an iterative spatial partitioning technique called a deep KD-tree, the model can accurately estimate probability distributions in high-dimensional systems even when experimental samples are scarce.

What Problem Does This Paper Address?

The Fokker-Planck (FP) equation is essential for describing how probability density functions (PDFs) evolve in systems influenced by random noise, such as chemical reactions, economic markets, or structural engineering. However, solving it is notoriously difficult due to two main hurdles:

  1. The Curse of Dimensionality: Traditional numerical methods (like finite element or finite difference) require a grid. As the number of dimensions increases, the number of grid points grows exponentially, making computation impossible for high-dimensional systems.
  2. The Normalization Constraint: A PDF must always integrate to exactly 1. In deep learning approaches, satisfying this "global" condition usually requires a massive amount of data to ensure the neural network doesn't "cheat" by finding a solution that looks right locally but fails the total probability test.

In real-world engineering, we rarely have the luxury of "big data." This paper tackles the challenge of solving these complex equations when only a small, random set of experimental data points is available.

Methods

The authors introduce the super-DL-FP framework, which merges two distinct concepts:

  • Deep Learning (DL-FP): A neural network is used as a "universal function approximator" to represent the PDF. Unlike grid-based methods, neural networks can calculate derivatives analytically, allowing them to handle off-grid points easily.
  • Deep KD-tree: This is the paper's core innovation. A standard KD-tree partitions space into "boxes" based on data points. The authors use an iterative "deep" version of this to estimate the "volume" associated with each random data point.
  • They use a "peeling the fruit" analogy: the algorithm identifies boundary points, separates them, and then re-partitions the interior points.
  • This allows the model to perform numerical integration on a sparse, non-uniform set of random points, providing a reliable way to enforce the normalization condition (the integral = 1) during training.

The final loss function used to train the network includes three parts: the FP equation itself, the boundary conditions, and the KD-tree-derived normalization integral.

Datasets

The researchers evaluated their method using synthetic "experimental" data generated via Stochastic Runge-Kutta (SRKII) simulations of known systems:

  • 2D Van der Pol System: A classic oscillating system used to test the basic accuracy of the PDF "crater" shape.
  • 4D Coupled System: A higher-dimensional system involving multiple variables and non-linear potentials.
  • 6D Stochastic System: A complex high-dimensional test case designed to prove the method can scale where traditional grid-based methods fail.

Key Insights and Ideas

  • Midpoint Integration: The authors found that using the geometric midpoint of the KD-tree partitions as the integral point (rather than the random data point itself) significantly improved stability.
  • Data Efficiency: The deep KD-tree acts as a bridge between discrete, random samples and continuous calculus. It "teaches" the neural network the geometry of the space it is operating in without needing a dense, structured grid.
  • Analytical Solutions: Because the output is a neural network, the solution is analytical. You can query the probability at any coordinate $(x, y, z...)$ instantly, whereas traditional methods only give you values at specific pre-defined grid points.

Key Results

The super-DL-FP method showed a clear advantage in accuracy-per-data-point:

  • Van der Pol (2D): Achieved 99.30% accuracy with 1,000 points and a respectable 84.21% accuracy with just 100 points.
  • Superior Accuracy: When compared at 200 calculation points, super-DL-FP achieved an error magnitude of $10^{-2}$, while traditional methods like Central Difference and Sylvester Flow were stuck at $10^{-1}$.
  • High-Dimensional Success: The model successfully solved the 6D system with 200,000 points—a feat that would require billions of points using standard grid-based discretization.

Limitations and Future Work

While the results are promising, the authors note a few areas for improvement:

  • Theoretical Proof: The "Deep KD-tree" approach is currently justified through visualization and empirical results; a rigorous mathematical proof of its convergence is still needed.
  • Integral Schemes: The current model uses a conservative "midpoint integral." The authors suggest that more sophisticated schemes, such as quantile integrals, might further improve accuracy.
  • Broad Application: The framework is currently tailored for the Fokker-Planck equation, but the authors believe the deep KD-tree logic could be applied to other integral-differential equations in physics and engineering.

Summary updated

Abstract

<title>Abstract</title> The Fokker-Planck (FP) equation can deterministically describe the evolution of the probability density function, which plays an extremely significant role in the fields of stochastic dynamics. Unfortunately, the limited samples that arise from the consideration of engineering practice are inevitable, which restricts the solving of the FP equation. Accordingly, in the present study, a super-DL-FP framework is established to solve the steady-state FP equation with a small amount of data, through combining the deep KD-tree and the DLFP approach proposed in [Chaos 30, 013133 (2020)]. It should be emphasized that the normalization condition is of great importance and have to be considered in solving the steady-state FP equation. An appropriate integral estimation for the normalization condition under non-uniform meshing can effectively improve the precision of the solution, but it is still a challenging problem, especially for the case of small data. Thus, the so-called deep KD-tree method is innovatively proposed to estimate the normalized integral with a small random dataset. The main target is to obtain the appropriate discrete integral points and corresponding integral volumes by executing multiple KD-tree segmentation based on random data on the integral region. Several numerical experiments and comparisons are implemented to illustrate the superior performance of the super-DL-FP method. The obtained results indicate that the proposed algorithm can accomplish higher accuracy in the sense of lower cost than the well-known algorithms like center difference scheme, Chebyshev spectrum algorithm, and normalized flow approach.
BibTeX
@inproceedings{zhang2022solving,
  author = {Hao Zhang and Yong Xu and Qi Liu and Xiaolong Wang and Yongge Li},
  title = {{Solving Fokker–Planck equations using deep KD-tree with a small amount of data}},
  booktitle = {Nonlinear},
  year = {2022},
  doi = {10.1007/s11071-022-07361-2},
}

Loading related content...

Figure