TL;DR
This paper introduces a novel method, NN-aPC, to quantify both parametric and approximation uncertainties in physics-informed neural networks (PINNs) for solving stochastic differential equations. By combining arbitrary polynomial chaos (aPC) expansions with dropout-based uncertainty estimation, the approach enables more robust predictions and an active learning strategy for sensor placement.
What Problem Does This Paper Address?
Traditional physics-informed neural networks (PINNs) are powerful for solving differential equations without explicit meshing, but they struggle with quantifying uncertainty. This uncertainty arises from two main sources: parametric uncertainty, stemming from randomness in the problem's parameters (e.g., material properties), and approximation uncertainty, due to the inherent limitations of the neural network architecture itself. Accurately capturing these uncertainties is crucial for reliable predictions, especially in scientific and engineering applications where understanding the confidence in a solution is as important as the solution itself. Existing PINNs often provide point estimates without a clear indication of their reliability.
Methods
The core of the proposed method, NN-aPC, lies in its hybrid approach:
- Arbitrary Polynomial Chaos (aPC) Expansion: This technique is used to represent the stochastic solution of the differential equation. Instead of directly learning the complex stochastic solution, the method learns the coefficients (modes) of its aPC expansion. This is advantageous because aPC provides an orthogonal basis that can represent correlated random fields without assumptions on data distribution.
- Neural Networks for Mode Learning: Separate neural networks (DNNs) are trained to learn these aPC modes. These networks are "physics-informed" in that their training incorporates the governing differential equations.
- Dropout for Approximation Uncertainty: To quantify the uncertainty in the DNN's approximation of these modes, the authors employ dropout. Traditionally used for regularization, dropout is adapted here to provide stochastic predictions during inference. By running the network multiple times with dropout enabled, the variance in predictions directly estimates the approximation uncertainty.
- Active Learning Strategy: The dropout-induced uncertainty is then used to guide an active learning strategy. This strategy intelligently suggests where to place new sensors to most effectively reduce the approximation uncertainty and improve the overall model accuracy.
Datasets
The paper utilizes synthetic data generated through Monte Carlo (MC) sampling for numerical experiments. Specific details on the number of snapshots and sensor configurations are provided for each example:
- Forward Problem (Poisson Equation): Data generated from 1000 MC snapshots of the forcing term $f(x; \omega)$ and corresponding solutions $u(x; \omega)$. 13 $f$-sensors and 2 $u$-sensors at boundaries were used for training.
- Inverse Problem (Elliptic Equation): Data generated from MC sampling of the diffusion coefficient $k(x; \omega)$ and corresponding solutions $u(x; \omega)$. Various sensor configurations (e.g., 4 $k$-sensors, 7 $u$-sensors) were used for training and testing.
Key Insights and Ideas
- Decomposing Uncertainty: The paper successfully separates and quantifies two distinct sources of uncertainty: parametric (from the problem itself) and approximation (from the model). This is a significant step towards more reliable uncertainty quantification in PINNs.
- Dropout as a Dual-Purpose Tool: Leveraging dropout not only for regularization but also for estimating approximation uncertainty is an elegant and computationally efficient approach. It avoids the higher computational cost of traditional Bayesian neural networks.
- Intelligent Sensor Placement: The active learning strategy, guided by dropout uncertainty, demonstrates a practical way to optimize data acquisition by placing sensors where they are most needed to reduce model uncertainty.
- aPC for Stochasticity: Using aPC expansions to represent stochastic solutions simplifies the learning problem by breaking down complex random behavior into learnable modes.
Key Results
- Accurate Predictions: The NN-aPC method demonstrated accurate predictions for both forward and inverse stochastic problems, closely matching reference solutions derived from a large number of MC samples.
- Effective Uncertainty Quantification: The method successfully quantified both parametric and approximation uncertainties, providing confidence intervals for predictions.
- Improved Accuracy with Active Learning: The active learning strategy, by strategically adding sensors, significantly reduced the prediction error. For instance, in one case, the relative error of $k$ predictions was reduced from over 5% to less than 1% with the deployment of only 8 new sensors.
- 2nd-Order aPC Advantage: Using a 2nd-order aPC expansion generally led to significant improvements in accuracy compared to a 1st-order expansion, especially for the inverse problem.
Limitations and Future Work
- High-Dimensional DNNs: The paper acknowledges that a high-dimensional random space can lead to high-dimensional DNN outputs, potentially making the training process more challenging.
- Future Research Directions: The authors suggest exploring other methods for dealing with high-dimensional stochastic problems, such as Generative Adversarial Networks (GANs), and plan to investigate these in future work. They also mention the potential of abandoning aPC and directly using stochastic data as input, though they highlight challenges in explicitly expressing stochasticity and marking random instances.
Summary updated