Abstract
Discovering governing equations from data is critical for scientific and engineering applications, yet existing methods struggle with data sparsity, noise, and high computational costs. We propose KBASS (Kernel learning and BAyesian Spike-and-Slab priors), a novel framework that combines flexible kernel regression with Bayesian variable selection. By leveraging tensor algebra and Kronecker product constructions, KBASS enables efficient posterior inference and uncertainty quantification. Our results across various ODE and PDE benchmarks demonstrate superior robustness to noise and significantly faster training times compared to state-of-the-art methods. The code is available at https://github.com/long-da/KBASS.
Introduction
In fields ranging from fluid dynamics to ecology, researchers often possess observational data but lack the exact mathematical equations—typically Ordinary Differential Equations (ODEs) or Partial Differential Equations (PDEs)—that describe the system's underlying physics. Discovering these "governing equations" is a fundamental challenge in data-driven science.
Current state-of-the-art methods face three major hurdles:
- Data Quality: Methods like SINDy rely on numerical differentiation, which is highly sensitive to noise and sparsity.
- Computational Cost: Physics-Informed Neural Networks (PINNs) avoid numerical differentiation but are notoriously difficult to tune and expensive to train.
- Uncertainty Quantification: Most methods provide a single "best guess" without indicating confidence, which is problematic in scientific applications where understanding the margin of error is critical.
Methodology: The KBASS Framework
KBASS addresses these limitations through three core technical pillars:
Kernel Regression for Function Estimation
Instead of using deep neural networks, KBASS employs kernel regression to estimate target functions and their derivatives. Kernels act as global interpolants that are naturally robust to noise, providing a smoother and more reliable approximation of the underlying physics than local numerical differentiation.
Bayesian Spike-and-Slab Priors
To identify the simplest governing equation, the model must select a few relevant terms from a large dictionary of candidates (e.g., $u, u^2, u_{xx}$). KBASS utilizes a spike-and-slab prior, the gold standard for Bayesian variable selection. Each term is treated as a binary choice: it is either included in the equation (the "slab") or effectively zero (the "spike"). This allows the model to calculate the posterior probability of each term's relevance, eliminating the need for arbitrary manual thresholding.
Computational Efficiency via Tensor Algebra
Kernel methods typically suffer from $O(N^3)$ complexity due to matrix inversions. KBASS overcomes this by placing function values on a Cartesian mesh. This structure allows for the use of Kronecker products and tensor algebra, reducing computational costs from cubic to linear relative to the mesh size. Posterior inference is performed efficiently using a specialized Expectation-Propagation Expectation-Maximization (EP-EM) algorithm.
Experimental Results and Discussion
The authors evaluated KBASS against benchmarks including the Van der Pol oscillator, the chaotic Lorenz 96 system, and PDEs such as Burger’s, Kuramoto-Sivashinsky (KS), and Allen-Cahn equations.
Robustness and Accuracy
KBASS demonstrates exceptional resilience to noise. While traditional methods often fail when noise levels exceed 5-10%, KBASS successfully recovered governing equations with up to 20% noise.
Threshold-Free Discovery and Uncertainty
A significant advantage of the Bayesian approach is the ability to identify terms with very small physical constants. For instance, in the Allen-Cahn equation, the diffusion rate is approximately $10^{-4}$. While threshold-based methods might discard such terms as noise, KBASS identifies them based on their high posterior probability.
Computational Speed
KBASS is consistently faster than competing Bayesian methods like BSL (1.9x to 7.3x faster). Compared to PINN-SR, the speedup is even more dramatic; on the KS equation, PINN-SR took 20 times longer and failed to converge to the correct solution.
Conclusion
KBASS provides a robust, efficient, and probabilistic framework for equation discovery. By integrating kernel learning with spike-and-slab priors, it handles sparse and noisy data where other methods fail. Currently, the method relies on a pre-specified dictionary of candidate operators. Future work may involve dynamically expanding this dictionary or incorporating symbolic regression to "invent" new terms, further reducing the reliance on prior domain knowledge.
Summary updated