PDE-State Conditioning for Learned Optimization Using Physics-Informed Neural Networks

Farzana Yasmin, Ricardo VilaltaarXiv 2026 · arXiv

TL;DR

This paper introduces PDE-state conditioning for meta-learned optimizers, allowing them to "see" the global physics state of a Physics-Informed Neural Network (PINN) during training. By modulating the optimizer’s internal representations with features like residual geometry, the authors achieve significantly faster convergence and better generalization across different types of partial differential equations (PDEs) compared to standard optimizers like Adam.

What Problem Does This Paper Address?

Training Physics-Informed Neural Networks (PINNs) is notoriously difficult. Unlike standard supervised learning, PINNs must satisfy a physics-based loss (the PDE residual) across a continuous domain. Standard optimizers like Adam treat every parameter update as a local gradient-descent problem, ignoring the global structure of the physics being solved.

While "learned optimizers" (LOs)—neural networks trained to act as optimization algorithms—have shown promise, they typically only observe local history (e.g., past gradients and moments). They remain "blind" to the actual physics state, such as whether the residual is concentrated in a specific area or how the operator's nonlinearity is behaving. This paper asks: Can we improve PINN training by giving the optimizer a global "weather report" of the current PDE state?

Methods

The authors propose a structured meta-learned optimizer that incorporates an 11-dimensional PDE state vector ($s_t$). This vector is divided into three functional groups:

  • G1 (Operator Structure): Static descriptors like derivative orders and nonlinearity.
  • G2 (Residual Geometry): Dynamic features capturing how the physics residual is distributed (concentration and "roughness").
  • G3 (Update Response): How the physics residuals reacted to the previous update.
The PDE-state-conditioned optimizer architecture uses a context network to modulate the local optimizer's hidden state via bounded FiLM.

To integrate this global information without overwhelming the local update logic, they use Bounded FiLM (Feature-wise Linear Modulation). A small context network maps the PDE state to scale ($\gamma$) and shift ($\beta$) parameters that adjust the optimizer's internal hidden layers. Crucially, these adjustments are "bounded" (using a tanh function) to ensure the global state guides the optimizer rather than completely overriding its local gradient-based logic.

Datasets

The researchers focused on one-dimensional time-dependent PDEs:

  1. Linear Advection (LA): Used for meta-training. It's a relatively simple equation modeling the transport of a quantity.
  2. Korteweg-de Vries (KdV): Used for evaluation (transfer). It introduces nonlinearity and third-order dispersion.
  3. Burgers’ Equation: Used for evaluation (transfer). It combines nonlinear transport with diffusion, often leading to sharp gradients (shocks) that are difficult to optimize.

Key Insights and Ideas

  • G2 is the "Aha!" Feature: Through ablation studies, the authors found that Residual Geometry (G2)—specifically how concentrated or "rough" the physics error is across the domain—was the most informative signal. Surprisingly, providing the full 11-dimensional state (All11) was often less effective than just using the 2-dimensional G2 state.
  • Conditioning vs. Direct Input: Simply appending the PDE state to the optimizer's input (a "NoContext" approach) performed poorly. The structured modulation provided by FiLM was essential for the optimizer to effectively use the global information.
  • Robustness through Meta-Learning: One of the biggest wins was reduced sensitivity. Learned optimizers can be finicky depending on their initialization. PDE conditioning significantly lowered the variance in performance across different meta-training runs, making the resulting optimizer much more reliable.
Comparison of different PDE-state feature groups. G2 consistently emerges as a strong performer across different equations.

Key Results

The PDE-state-conditioned optimizer (specifically the G2+FiLM variant) outperformed both the standard Adam optimizer and an unconditioned learned optimizer across all tested metrics:

  • Accuracy Gains: On the Linear Advection task, it reduced the relative $L_2$ error by 74.3% compared to Adam.
  • Zero-Shot Transfer: Even though it was only trained on Linear Advection, it generalized to the more complex KdV and Burgers equations, reducing relative $L_2$ error by 30.0% and 7.3% respectively over Adam.
  • Physics Satisfaction: The "Physics MSE" (how well the solution actually obeys the PDE) was consistently lower, indicating a more physically faithful solution.
Quantitative performance comparison across LA, KdV, and Burgers equations. The conditioned optimizer (bottom row) shows the best results.

Limitations and Future Work

  • Dimensionality: The study was limited to 1D benchmarks. Extending this to 2D or 3D problems would require more complex "probes" to capture residual geometry efficiently.
  • Source Diversity: The optimizer was meta-trained only on Linear Advection. The authors suggest that training on a broader family of PDEs might lead to even more robust universal optimizers.
  • Feature Engineering: While G2 was effective, the authors note that "simply adding more residual or PDE-state information is not automatically beneficial," suggesting a need for more automated feature discovery in the future.

Summary updated

Abstract

Physics-informed neural network (PINN) optimization depends on global properties of the underlying PDE and how the physics residual evolves across the domain. Standard optimizers such as Adam use fixed gradient-based statistics, while learned PINN optimizers replace the fixed rule with a learned update built from local parameters and optimization-history features. However, none explicitly use PDE-level information. We study two coupled design questions: what PDE-level information should a learned optimizer observe, and how should that information influence its update? We introduce an 11-dimensional PDE state-capturing operator structure, residual geometry, and short-horizon update response, and use it to condition a structured meta-learned optimizer through bounded feature-wise modulation. Repeated design studies identify the compact residual-geometry state G2 and bounded FiLM as the strongest overall choices among those tested. The resulting PDE-State-conditioned optimizer improves all evaluation metrics (relative-L2, L1, solution MSE, and physics MSE) over Adam and the unconditioned learned optimizer on Linear Advection, KdV, and Burgers.
BibTeX
@misc{yasmin2026pde,
  author = {Farzana Yasmin and Ricardo Vilalta},
  title = {{PDE-State Conditioning for Learned Optimization Using Physics-Informed Neural Networks}},
  year = {2026},
  eprint = {2611.25185},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25185},
}
Figure