Abstract
Physics-Informed Neural Networks (PINNs) have emerged as a powerful paradigm for solving partial differential equations (PDEs) by integrating physical laws into the loss function of deep learning models. However, training PINNs often involves a complex multi-objective optimization problem where the PDE residual, initial conditions (IC), and boundary conditions (BC) compete for gradient priority. This "tug-of-war" frequently leads to convergence at local minima or physically inconsistent solutions. This paper introduces DB-PINN (Dual-Balanced PINN), a novel hierarchical weighting scheme designed to stabilize training by balancing the PDE residual against the aggregate of all conditions (inter-balancing) and subsequently distributing weight based on the learning difficulty of specific conditions (intra-balancing).
Introduction
Training a PINN is fundamentally a multi-objective optimization task. The objective function typically consists of a weighted sum of the PDE residual loss and various constraint losses (IC, BC, and data). In standard "vanilla" PINNs, these terms are often weighted equally. However, the PDE residual loss frequently dominates the gradient flow, causing the network to satisfy the differential equation while failing to adhere to the specific boundary or initial conditions. This results in solutions that are mathematically valid in isolation but physically incorrect for the specific system being modeled.
While existing Gradient Weighting (GW) methods attempt to address this by balancing pairwise relationships (e.g., PDE vs. BC), they often overlook the internal dynamics between different constraints. DB-PINN addresses this gap by introducing a two-tier weighting system that accounts for both the global relationship between physics and constraints and the relative difficulty of individual constraints.
Methodology: The DB-PINN Framework
DB-PINN operates through three primary mechanisms designed to ensure stable and accurate convergence:
1. Inter-Balancing
This stage focuses on the "big picture" by calculating an aggregated weight for all condition-fitting losses relative to the PDE residual. By utilizing gradient statistics—such as mean magnitude, standard deviation, or kurtosis—the framework ensures that the PDE residual does not overwhelm the other constraints during backpropagation. This prevents the model from ignoring the boundary conditions that define the unique solution to the PDE.
2. Intra-Balancing
The novel "inner" tier of the framework evaluates a difficulty index ($I_t$) for each specific condition. The difficulty is defined as the ratio of the current loss ($L_t$) to its historical average ($\mu_{L_t}$):
$$I_t = \frac{L_t}{\mu_{L_t}}$$
If a specific boundary condition exhibits a high loss relative to its history, the system identifies it as "difficult" and assigns it a larger share of the aggregated weight budget. This dynamic pivoting allows the model to focus its learning capacity on the most challenging aspects of the physical system in real-time.
3. Robust Weight Updates via Welford’s Algorithm
Gradient-based weights are notoriously volatile, often producing spikes that can destabilize training. Rather than relying on standard Exponential Moving Averages (EMA), which require sensitive hyperparameter tuning, DB-PINN employs Welford’s online algorithm. This provides a stable, tuning-free method for tracking loss means and updating weights smoothly, effectively preventing arithmetic overflow and numerical instability.
For systems with only a single type of condition, the authors propose Inner-Balancing, which decomposes a single condition into sub-components (e.g., different spatial axes) to apply the same hierarchical logic.
Experimental Evaluation
The researchers evaluated DB-PINN against several challenging PDE benchmarks known for being difficult for standard PINN architectures:
- Klein-Gordon Equation: A nonlinear equation central to quantum mechanics.
- Wave Equation: A "stiff" 1D version that typically causes standard PINNs to fail.
- Helmholtz Equation: A time-independent PDE used in acoustics and electromagnetics.
Key Results
DB-PINN consistently outperformed standard PINNs and state-of-the-art weighting methods, such as Uncertainty Weighting and SA-PINN, across all benchmarks:
- Accuracy: On the Klein-Gordon equation, DB-PINN achieved a Relative L2 Error (L2RE) of 0.636%, a significant improvement over the 7.272% error seen with equal weighting and the 1.262% error of the best previous gradient-weighting methods.
- Stiff Systems: While standard PINNs often fail the Wave Equation (errors ~40%), DB-PINN reduced the error to 0.292%.
- Convergence: DB-PINN demonstrated faster convergence and smoother loss curves, reaching lower error thresholds in fewer iterations than its competitors.
Discussion and Future Work
The primary contribution of DB-PINN is the shift from pairwise balancing to a holistic-internal balancing strategy. By treating the constraints as a collective "budget" that is then distributed based on dynamic difficulty, the model avoids overfitting easy conditions while neglecting hard ones. Furthermore, the integration of Welford’s algorithm addresses the practical engineering challenge of weight stability.
Future research directions include:
- Statistic Selection: Investigating how to automatically select the optimal gradient statistic (mean vs. kurtosis) for different PDE types.
- High-Dimensional Scaling: Testing the intra-balancing logic on high-dimensional problems, such as the Schrödinger equation in many-body systems.
- Inverse Problems: Applying the dual-balancing framework to parameter identification tasks where the physical constants themselves are unknown.
Summary updated