Discovering the underlying differential equations that govern a dynamical system directly from observational data is a core objective of modern data-driven science. However, moving this pipeline from pristine numerical simulations to real-world applications introduces a fundamental obstacle: observational noise. As emphasized by Kurths et al., real-world data is frequently contaminated by measurement errors and stochastic perturbations that obscure the true mathematical laws.
To understand why noise poses such a severe challenge, we must look at how standard equation discovery works, why it fails, and the core mechanisms researchers are using to make it robust.
The Fundamental Vulnerability: Noise Amplification
The most common paradigm for equation discovery—exemplified by frameworks like the Sparse Identification of Nonlinear Dynamics (SINDy)—formulates the problem as a sparse linear regression:
$$\dot{\mathbf{x}}(t) = \mathbf{\Theta}(\mathbf{x}(t))\mathbf{\Xi}$$
where $\mathbf{\Theta}(\mathbf{x})$ is a library of candidate nonlinear functions (e.g., polynomials, trigonometric functions) and $\mathbf{\Xi}$ is a sparse matrix of coefficients to be determined.
This formulation relies on having accurate estimates of both the state $\mathbf{x}(t)$ and its time derivative $\dot{\mathbf{x}}(t)$. In practice, we rarely measure derivatives directly; we must compute them numerically from discrete, noisy measurements of $\mathbf{x}(t)$.
This is where the mathematics breaks down. Numerical differentiation acts as a high-pass filter. If a continuous signal is corrupted by even a small amount of high-frequency white noise, taking the derivative drastically amplifies that noise. The resulting derivative estimates become highly volatile, causing sparse regression algorithms to select incorrect terms or fail entirely. This sensitivity to noise is a shared bottleneck across many data-driven dynamical systems methods, including standard Dynamic Mode Decomposition (Baddoo et al.; Takeishi et al.) and geometric causal discovery methods like Convergent Cross Mapping (Liu et al.).
To overcome this, recent research focuses on three main pillars: surrogate-mediated derivative estimation, integral/time-evolution reformulations, and structural/probabilistic priors.
Mechanism 1: Surrogate-Mediated Smoothing
Instead of differentiating noisy data directly, a prominent strategy inserts a smooth, differentiable surrogate model between the raw data and the symbolic discovery step. As Fallah Mortezanejad et al. observe, traditional symbolic regression methods frequently fail when applied directly to high-noise time series; they require a surrogate model to smooth the data and provide stable derivatives before a coherent mathematical structure can be identified.
Noisy Data ──> [ Smooth Surrogate Model ] ──> Stable Derivatives ──> Sparse Regression ──> Discovered Equation
Neural networks are exceptionally well-suited for this role due to their implicit regularization properties. Ramasinghe et al. demonstrate that coordinate-based Multi-Layer Perceptrons (MLPs) possess powerful "neural priors" that naturally bias them toward learning smooth, low-frequency signals before fitting high-frequency noise. By training a network to fit the noisy trajectory, the network acts as an optimized low-pass filter. Once trained, the analytical or automatic derivatives of the network can be queried safely.
Taking this concept further, Kaheman et al. developed an architecture that uses automatic differentiation to simultaneously denoise time-series data, learn the underlying system dynamics, and explicitly characterize the probability distribution of the measurement noise. Similarly, Yang et al. embed numerical ordinary differential equation (ODE) solvers directly into a neural network framework (ODENet). By optimizing the network to match the global trajectory rather than point-wise derivatives, the framework filters out stochastic fluctuations and uncovers explicit ODE coefficients even under significant white noise.
Mechanism 2: Reformulating the Objective via Time Evolution
A second core mechanism bypasses numerical differentiation entirely by changing the optimization objective from differential matching to integral matching. If differentiation amplifies noise, integration smooths it out.
Kang et al. introduce this concept through the IDENT framework, which evaluates candidate partial differential equations (PDEs) using a Time Evolution Error (TEE) metric. Instead of checking if the library terms match a noisy derivative at a single point, IDENT integrates the candidate PDE schemes forward in time. Because the integration accumulates data over a temporal window, zero-mean Gaussian noise naturally cancels out, allowing the framework to identify equations accurately even with substantial data corruption.
A related mathematical reformulation addresses implicit dynamics (equations where derivatives appear nonlinearly or on both sides of the equal sign). Traditional methods for implicit discovery rely on null-space calculations, which are highly unstable when data is noisy. To resolve this, Kaheman et al. developed SINDy-PI, which reformulates the implicit optimization problem to isolate and evaluate candidate terms individually. By avoiding global null-space calculations, the algorithm achieves an order-of-magnitude improvement in noise tolerance.
Mechanism 3: Constraining the Search Space with Physics and Sparsifying Priors
When data is highly corrupted, an unconstrained optimization algorithm can easily overfit the noise by combining spurious terms from the library. To prevent this, researchers inject structural constraints that restrict the space of permissible equations to those that are physically plausible or mathematically sparse.
Hard Physical Constraints
If we already know certain physical laws that the system must obey (such as energy conservation, incompressibility, or symmetries), we can force the optimization to respect them. Baddoo et al. introduce physics-informed dynamic mode decomposition (piDMD), which mathematically constrains the learned linear operators to belong to specific matrix groups (e.g., unitary, symplectic) representing these physical invariants. This constraint prevents the model from interpreting noise as a violation of physics. In the nonlinear regime, Du et al. combine reinforcement learning with symbolic mathematics to discover open-form PDEs, embedding these discovered constraints directly into neural networks to maintain stability under extreme noise levels up to 125%.
Probabilistic and Sparsifying Priors
Instead of hard constraints, Bayesian frameworks use probabilistic priors to penalize overly complex models and quantify the uncertainty introduced by noise.
Noisy Data + Sparsifying Prior (e.g., Laplacian) ──> [ Bayesian Inference ] ──> Probability Distribution of Coefficients
Hirsh et al. implement this in UQ-SINDy, utilizing sparsity-promoting priors to compute a posterior probability distribution over the library coefficients. This allows the model to output not just a single deterministic equation, but the exact probability of each term's inclusion. Similarly, Mabrouk et al. leverage a Laplacian prior within a sparse Bayesian inference framework to isolate deterministic drift from stochastic noise in stochastic differential equations (SDEs). When the system's parameters vary over time, Voina et al. demonstrate that deep generative models can successfully track these non-stationary variations while maintaining robustness against additive Gaussian noise.
By shifting from local, noise-sensitive operations (like numerical differentiation) to global, regularized operations (like surrogate smoothing, temporal integration, and Bayesian priors), modern equation discovery frameworks can reliably extract the true mathematical laws governing complex dynamical systems from messy, real-world observations.
Key Foundational Work
Discovering governing equations from data by sparse identification of nonlinear dynamical systems (2016) — contributed 1 concept elements to this direction.
Single-shooting homotopy method for parameter identification in dynamical systems (2012) — contributed 1 concept elements to this direction.