DeltaPhi: Physical States Residual Learning for Neural Operators in Data-Limited PDE Solving

Xihang Yue, Yi Yang, Linchao ZhuNeurIPS 2025 · OpenReview

TL;DR

DeltaPhi is a novel framework that tackles the data-scarcity problem in neural PDE solving by teaching models to learn the residuals between similar physical states rather than direct input-output mappings. By leveraging the inherent stability of physical systems, it provides "implicit data augmentation" that significantly improves accuracy and generalization across various architectures and complex geometries.

The Challenge of Data-Limited PDE Solving

Partial Differential Equations (PDEs) are the language of the universe, describing everything from fluid flow to heat transfer. While traditional numerical solvers are accurate, they are computationally expensive. Machine learning—specifically neural operators like the Fourier Neural Operator (FNO)—offers a faster alternative by learning to map input parameters (like initial conditions) directly to solutions.

However, these models are notoriously data-hungry. Generating high-fidelity training data requires expensive simulations or physical experiments. When data is scarce, neural operators struggle to generalize and often succumb to "distribution bias," where the model memorizes specific training samples rather than learning the underlying physics. The authors identify a critical gap: current models treat every training sample as an isolated point, ignoring the relational structure between similar physical states.

Methods: Learning the Residual

The core innovation of DeltaPhi is a fundamental shift in perspective. Instead of asking a model to predict a solution $u$ from scratch, it asks the model to predict the difference between a new input and a known "neighbor" from the training set.

DeltaPhi Architecture Comparison
  1. Residual Reformulation: For a given input $a_i$, the framework retrieves a similar auxiliary sample $(a_k, u_k)$ from the training set. The model then learns a residual mapping $G^\Delta$ such that the final prediction is $\hat{u}_i = G^\Delta(a_i, a_k) + u_k$.
  2. Auxiliary Sample Retriever: During training, the system uses cosine similarity to find the top-$K$ most similar samples in the training set and randomly picks one. This creates a powerful form of implicit data augmentation: a single training sample can be paired with many different neighbors, forcing the model to learn a diverse range of residuals.
  3. Architecture Agnostic: DeltaPhi isn't a standalone new model; it’s a versatile wrapper. It can be seamlessly integrated into existing architectures (FNO, DeepONet, GNOT, etc.) by simply concatenating the target input with the auxiliary input and adding a physical residual connection at the output layer.
  4. Cross-Resolution Alignment: To handle cases where training and testing data have different resolutions, the authors use Fourier-based downsampling/upsampling to ensure the target and auxiliary samples are mathematically compatible.

Key Insights

  • Stability as a Prior: The method works because most physical systems are stable—small changes in initial conditions lead to small changes in the trajectory. DeltaPhi bakes this physical intuition directly into the learning process.
  • From Points to Tangent Vectors: Geometrically, direct learning tries to map isolated points on a solution manifold. DeltaPhi learns "tangent vectors" (the residuals) that describe the manifold's local geometry, providing a much richer learning signal.
  • Mitigating Overfitting: Because the "reference" auxiliary sample changes randomly during training, the model cannot simply memorize a fixed mapping. It must understand how the solution changes relative to different baselines.

Key Results

DeltaPhi demonstrated consistent and often dramatic improvements over "direct" learning baselines across a rigorous suite of benchmarks, including standard 2D flows and complex 3D meshes.

Error Reduction Visualizations
  • Huge Gains in Complex Physics: On irregular domain problems like Pipe Turbulence and Heat Transfer, DeltaPhi reduced relative error by 40% to 50% compared to the state-of-the-art NORM model.
  • Data Efficiency: A model trained with DeltaPhi on only 500 samples often outperformed a standard model trained on 700 or even 900 samples.
  • Resolution Generalization: In "zero-shot" scenarios (training on low-res, testing on high-res), DeltaPhi significantly mitigated the performance drop typically seen in FNO-based models.
  • Robustness: Statistical analysis showed that the model is remarkably robust to the choice of the auxiliary sample; even if the retriever doesn't find the "perfect" neighbor, the residual learning still provides a measurable benefit.

Limitations and Future Work

While powerful, the authors note a few areas for future exploration. In highly chaotic systems (like very low-viscosity fluids), small initial differences can lead to wildly different outcomes over time, making simple similarity metrics less effective. The current version relies on cosine similarity, but more advanced, physics-aware similarity metrics (perhaps learned in a latent space) could further unlock performance. Additionally, explicitly incorporating boundary information into the retriever could be a promising next step for handling complex boundary conditions.

Summary updated

Abstract

The limited availability of high-quality training data poses a major obstacle in data-driven PDE solving, where expensive data collection and resolution constraints severely impact the ability of neural operator networks to learn and generalize the underlying physical system. To address this challenge, we propose DeltaPhi, a novel learning framework that transforms the PDE solving task from learning direct input-output mappings to learning the residuals between similar physical states, a fundamentally different approach to neural operator learning. This reformulation provides implicit data augmentation by exploiting the inherent stability of physical systems where closer initial states lead to closer evolution trajectories. DeltaPhi is architecture-agnostic and can be seamlessly integrated with existing neural operators to enhance their performance. Extensive experiments demonstrate consistent and significant improvements across diverse physical systems including regular and irregular domains, different neural architectures, multiple training data amount, and cross-resolution scenarios, confirming its effectiveness as a general enhancement for neural operators in data-limited PDE solving.
BibTeX
@inproceedings{yue2025deltaphi,
  author = {Xihang Yue and Yi Yang and Linchao Zhu},
  title = {{DeltaPhi: Physical States Residual Learning for Neural Operators in Data-Limited PDE Solving}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=ppOCvEonKT},
}
Figure