TL;DR
PINNeval introduces a rigorous, four-component evaluation standard for Physics-Informed Neural Networks (PINNs) to move the field beyond "vanilla" baselines. Accompanied by the PINNpipe JAX library, it demonstrates that many "state-of-the-art" methods lose their edge when tested within competitive pipelines, while also proving that quasi-Newton optimizers are far more practical and powerful for complex PDEs than previously claimed.
What Problem Does This Paper Address?
Physics-Informed Neural Networks (PINNs) are a powerful tool for solving partial differential equations (PDEs) without meshes, but they are notoriously difficult to train. The literature is currently flooded with new methods for "loss balancing" or "adaptive sampling," yet most are evaluated in isolation against weak, "vanilla" baselines.
The authors argue that this creates a false sense of progress. A method that looks great when added to a basic Multilayer Perceptron (MLP) trained with Adam might actually perform worse than existing techniques when used in a modern, highly optimized pipeline. Furthermore, there is no standardized way to report these configurations, making it nearly impossible to compare results across different research papers.
Methods
The authors propose a systematic decomposition of the PINN training pipeline into four essential components:
- Domain-Importance Adaptation: Deciding where to sample points or how to weight them (e.g., focusing on high-residual areas).
- Loss Balancing: Weighting the different terms in the loss function (e.g., PDE residuals vs. boundary conditions).
- Neural Network Architecture: The structure of the surrogate model (e.g., MLPs, PirateNets).
- Optimization Scheme: The choice of optimizers and their schedules (e.g., Adam, SOAP, L-BFGS).
To fix the evaluation crisis, they introduce PINNeval, a protocol that requires researchers to:
- Use competitive surrounding components (don't compare your new method against a "straw man" baseline).
- Report runtime alongside iteration counts, as some "efficient" methods are computationally expensive per step.
- Use a standardized EvalCard to document every hyperparameter and hardware detail for reproducibility.
They also release PINNpipe, a modular JAX-based framework that implements 18 popular methods and 9 benchmarks, allowing researchers to swap components in and out easily.
Datasets
The paper utilizes a suite of nine benchmark PDE problems that range from low-dimensional to chaotic and multi-scale systems:
- Standard Benchmarks: Allen-Cahn, Burgers, Wave, and Korteweg–de Vries (KdV).
- Complex/Chaotic Systems: Kuramoto–Sivashinsky (KS), Ginzburg–Landau, and Rayleigh–Taylor instability.
- High-Dimensional: N-dimensional Heat and Poisson equations (tested at 5+1 dimensions).
Key Insights and Ideas
- The "Competitive Baseline" Reality Check: Many specialized domain-adaptation methods (like RAD or R3) actually struggle to outperform simple random sampling when the rest of the pipeline is strong.
- Quasi-Newton Methods are Practical: Recent literature suggested that second-order (quasi-Newton) methods were too memory-intensive or "impractical" for complex PDEs. This paper refutes that, showing that a two-stage approach (first-order SOAP followed by second-order SSBroyden) is a "silver bullet" for accuracy.
- Bottlenecks are often in the Pipeline, not the Model: Researchers often blame a network's "expressivity" for failure, but the authors show that using the exact same small MLP with a better optimizer can reduce error by four orders of magnitude.
- No Single Winner in Loss Balancing: While GradNorm-based balancing is generally robust, the "Fixed" weight baseline remains surprisingly competitive, and some advanced methods (like SoftAdapt) can suffer from total robustness failures on certain problems.
Key Results
The most striking result is the comparison between a "modern" first-order pipeline (P1) and the authors' proposed two-stage pipeline (P2).
As shown in the table above, the P2 pipeline (MLP + SOAP + SSBroyden in float64) consistently achieves significantly lower relative $L^2$ errors—often by several orders of magnitude—while being drastically faster in terms of wall-clock time. For the Wave equation, P2 is roughly 24x faster than the PirateNet-based P1.
In their domain-adaptation study, they found that while Langevin Adaptive Sampling (LAS) showed promise, simple Random-R sampling remained the most robust choice across the entire benchmark suite.
Limitations and Future Work
- Scope: The current study focuses on forward problems. The authors acknowledge that inverse problems and operator learning (like DeepONets) share similar pipeline structures and plan to extend the protocol to those areas.
- Geometry: The benchmarks currently focus on regular domains (boxes, spheres). Evaluating performance on complex, irregular geometries remains a future challenge.
- Moving Target: Because "competitive best practice" evolves, the authors emphasize that the PINNeval reference instantiation must be updated regularly by the community.
Summary updated