Robust Identifiability for Symbolic Recovery of Differential Equations

Hillary Hauger, Philipp Scholl, Gitta Kutyniok

This paper, titled "Robust Identifiability for Symbolic Recovery of Differential Equations," addresses a critical gap in the field of automated scientific discovery: how to tell if the physical law you’ve "discovered" from noisy data is actually unique.

TL;DR

The authors develop a mathematical framework and two new algorithms (NR-FRanCo and NR-JRC) to determine the "identifiability" of partial differential equations (PDEs) in the presence of noise. By establishing rigorous error bounds for singular values and finite differences, the paper provides a way to verify if a discovered symbolic model is the unique solution or if the data is too noisy to draw a reliable conclusion.

What Problem Does This Paper Address?

In recent years, machine learning has moved from just predicting data to "discovering" the underlying physics—finding the symbolic equations (like $u_t = \alpha u_{xx}$) that govern a system. However, a major theoretical hurdle is identifiability: given a set of data, is there only one equation that fits it, or could many different equations be equally valid?

Previous research into identifiability mostly assumed "perfect," noise-free data. In the real world, data is always messy. Noise can make a unique system appear non-unique or lead an algorithm to "discover" a law that is mathematically unjustifiable. This paper tackles the challenge of identifying the uniqueness of PDEs when the input data is contaminated with noise.

Methods

The authors build upon existing frameworks (specifically work by Scholl et al., 2023) and extend them using classical numerical analysis tools:

  1. Error Bounding: They use Weyl’s Theorem to bound how noise affects the singular values of data matrices and derive new error bounds for Finite Differences (FD) when calculating derivatives from noisy points.
  2. NR-FRanCo (Noise Robust Stable Feature Rank Computation): Designed for linear PDEs. It examines the ratio of the smallest to the largest singular value of a feature matrix. If this ratio falls below a specific noise-dependent threshold, the PDE is flagged as non-unique.
  3. NR-JRC (Noise Robust Jacobian Rank Computation): Designed for algebraic and analytic PDEs. This is more complex as it involves calculating the Jacobian of the features, requiring higher-order derivatives which are notoriously sensitive to noise.
  4. Decision Thresholds: Instead of a binary "unique/not unique" guess, the algorithms provide a "no definitive conclusion" zone when noise levels are too high to make a mathematically sound judgment.

Datasets

The study uses simulated datasets where the ground-truth PDE is known, allowing the authors to test the algorithms' accuracy across varying noise levels ($\alpha$ ranging from $10^{-10}$ to $10^{-1}$):

  • Linear PDEs: Functions involving exponentials and trigonometric terms (e.g., $u = \exp(ax+t)$).
  • Algebraic/Analytic PDEs: More complex structures involving inverses, square roots, and arc-functions (e.g., $u = (x+t)^{-1}$).
  • Noise Model: Gaussian noise was added to the clean signal to simulate real-world sensor error.

Key Insights and Ideas

  • Uniqueness is easier to prove than non-uniqueness: The authors found that classifying a PDE as unique is much more robust against noise than proving it is non-unique. This is because the smallest singular value (used to detect non-uniqueness) is extremely sensitive to even tiny perturbations.
  • The Cost of Derivatives: NR-JRC is significantly less stable than NR-FRanCo. Because NR-JRC requires calculating the Jacobian of the features, it involves more derivative steps, which amplifies noise exponentially.
  • Ill-posedness vs. Algorithmic Failure: The paper argues that when an algorithm fails to reach a conclusion, it’s often not a "bug" in the code, but a reflection of the fact that the problem has become mathematically ill-posed due to excessive noise.

Key Results

  • NR-FRanCo Performance: For linear PDEs, the algorithm correctly identified unique equations even at high noise levels (up to $10^{-1}$). However, identifying non-uniqueness became unreliable once noise exceeded $10^{-5}$.
  • NR-JRC Performance: This method was much more fragile. While it could identify unique algebraic PDEs up to $10^{-6}$ noise, it struggled with non-uniqueness at noise levels as low as $10^{-8}$.
  • Finite Difference Order: Increasing the order of the finite difference scheme generally improved the certainty of the results, provided the noise was within a manageable range.

Limitations and Future Work

  • Unknown Constants: The mathematical bounds derived in the paper depend on several constants (like the maximum value of high-order derivatives) that are often unknown in practice and must be estimated from the data itself.
  • Deterministic vs. Probabilistic: The current framework uses deterministic "worst-case" bounds. The authors suggest that moving to probabilistic bounds (specifically tailored for Gaussian noise) could lead to tighter, more practical thresholds.
  • Scaling: While tested on 1D and 2D problems, scaling these robust checks to high-dimensional systems remains a challenge due to the "curse of dimensionality" in derivative calculation.

Summary updated

Abstract

Recent advancements in machine learning have transformed the discovery of physical laws, moving from manual derivation to data-driven methods that simultaneously learn both the structure and parameters of governing equations. This shift introduces new challenges regarding the validity of the discovered equations, particularly concerning their uniqueness and, hence, identifiability. While the issue of non-uniqueness has been well-studied in the context of parameter estimation, it remains underexplored for algorithms that recover both structure and parameters simultaneously. Early studies have primarily focused on idealized scenarios with perfect, noise-free data. In contrast, this paper investigates how noise influences the uniqueness and identifiability of physical laws governed by partial differential equations (PDEs). We develop a comprehensive mathematical framework to analyze the uniqueness of PDEs in the presence of noise and introduce new algorithms that account for noise, providing thresholds to assess uniqueness and identifying situations where excessive noise hinders reliable conclusions. Numerical experiments demonstrate the effectiveness of these algorithms in detecting uniqueness despite the presence of noise.
BibTeX
@inproceedings{hauger2025robust,
  author = {Hillary Hauger and Philipp Scholl and Gitta Kutyniok},
  title = {{Robust Identifiability for Symbolic Recovery of Differential Equations}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10887720},
}
Figure