Abstract
We consider the inverse problem for the Partial Differential Equations (PDEs) such that the parameters of the dependency structure can exhibit random changepoints over time. This can arise, for example, when the physical system is either under malicious attack or subject to extreme external conditions. For that purpose, we employ Physics Informed Neural Networks (PINNs) -- universal approximators that can incorporate prior information from any physical law described by a system of PDEs. This prior knowledge acts in the training of the neural network as a regularization that limits the space of admissible solutions and increases the correctness of the function approximation. We show that when the true data generating process exhibits changepoints in the PDE dynamics, this regularization can lead to a complete miss-calibration and a failure of the model. Therefore, we propose an extension of PINNs using a Total-Variation penalty which accommodates (multiple) changepoints in the PDE dynamics. These changepoints can occur at random locations over time, and they are estimated together with the solutions. We propose an additional refinement algorithm that combines changepoints detection with a reduced dynamic programming method that is feasible for the computationally intensive PINNs methods, and we demonstrate the benefits of the proposed model empirically using examples of different equations with changes in the parameters. In case of no changepoints in the data, the proposed model reduces to the original PINNs model. In the presence of changepoints, it leads to improvements in parameter estimation, better model fitting, and a lower training error compared to the original PINNs model.
Introduction
Physics-Informed Neural Networks (PINNs) have emerged as a powerful tool for solving inverse problems in physical systems by embedding Partial Differential Equations (PDEs) directly into the neural network's loss function. This approach allows researchers to estimate hidden physical parameters from observed data. However, standard PINNs operate under a rigid assumption: the physical constants (such as viscosity or thermal conductivity) remain constant over time.
In real-world scenarios—ranging from structural failures in engineering to sudden environmental shifts—these parameters often exhibit "changepoints." When these abrupt shifts occur, the global physics regularization of a standard PINN becomes a double-edged sword; it attempts to enforce a single physical law across the entire domain, leading to "smeared" transitions or total model failure. This paper introduces CP-PINNs, a framework designed to detect and adapt to these non-stationary physical environments.
Methodology
CP-PINNs introduce two primary innovations to the standard PINN architecture to handle temporal discontinuities:
- Total-Variation (TV) Regularization: To allow for abrupt jumps in physical parameters ($\lambda$), the authors incorporate a TV penalty into the loss function. Because TV is a sparsity-inducing norm, it encourages the model to keep parameters constant for long stretches while permitting sharp "jumps" at specific locations. A U-shaped weighting function is also employed to maintain stability near the temporal boundaries.
- Online Optimized Loss Weighting: Balancing the data-fitting loss, physics loss, and TV penalty is notoriously difficult. CP-PINNs utilize an Exponentiated Descent method—an online learning algorithm that dynamically re-weights these loss terms during training. This treats weight selection as a regret-minimization problem, ensuring the model prioritizes changepoint discovery early in the optimization process.
Experimental Results
The model was evaluated using synthetic datasets from classic physical benchmarks, including the 1D Advection-Diffusion and 2D Navier-Stokes equations, modified to include temporal jumps in parameters.
In the 1D Advection-Diffusion test, CP-PINNs identified changepoints at 0.334s and 0.670s, nearly matching the true values of 0.333s and 0.666s. As shown in Figure 1, standard PINNs failed to capture these transitions accurately. The right panel demonstrates how the online weight optimizer stabilizes the training process, eventually settling on balanced weights once the parameters are correctly identified.
Beyond parameter estimation, CP-PINNs maintain high fidelity in reconstructing the latent physical solution ($u$). As seen in the heatmaps in Figure 2, the estimated solution closely tracks the exact solution across the entire spatio-temporal domain, with minimal Mean Square Error (MSE) even at the points of transition.
Conclusion
CP-PINNs provide a robust extension to the PINN framework for systems with non-stationary dynamics. By combining TV regularization with dynamic weight optimization, the model simultaneously solves the underlying physics and identifies critical changepoints. Future research will focus on extending this framework to spatial changepoints and more complex, high-dimensional dynamical systems.
Summary updated