Towards fast weak adversarial training to solve high dimensional parabolic partial differential equations using XNODE-WAN

Hao Ni, Paul Valsecchi Oliva, Yue Wu, Cuiyu He, Hao Ni 0001Journal of Computational Physics 2022 · DOI

Abstract

Solving high-dimensional parabolic partial differential equations (PDEs) has long been hindered by the "curse of dimensionality." Traditional mesh-based methods scale exponentially with spatial variables, while early deep learning approaches often fail to distinguish the unique role of time in physical systems. This paper introduces XNODE-WAN, a hybrid framework that combines Weak Adversarial Networks (WAN) with Extended Neural Ordinary Differential Equations (XNODEs). By treating the PDE solution as an ODE evolving in time for fixed spatial coordinates, XNODE-WAN achieves up to a 99.8% reduction in training time compared to state-of-the-art methods and scales linearly up to 64 dimensions.

Introduction

Parabolic PDEs are essential for modeling phenomena such as heat conduction, image denoising, and derivative pricing in finance. Despite their importance, solving them in high-dimensional spaces remains a significant computational challenge. While mesh-free deep learning approaches like the Weak Adversarial Network (WAN) have made strides, they typically treat time ($t$) and space ($x$) as identical input coordinates within a standard Multi-Layer Perceptron (MLP). This approach ignores the inherent temporal flow of parabolic systems, leading to slow convergence and high data requirements.

Methodology

The XNODE-WAN Architecture

XNODE-WAN addresses the limitations of standard WANs by introducing a specialized architecture that incorporates physical inductive biases. The framework is built on two primary components:

  1. Adversarial Training (WAN): The system employs a minimax game between a primal network (the solution approximator) and an adversarial network (the test function). The adversarial network identifies regions where the primal solution fails to satisfy the weak form of the PDE.
  2. Extended Neural ODEs (XNODE): Instead of an MLP, the primal network uses Neural ODEs. For any fixed spatial point $x$, the solution is modeled as an ODE evolving over time. The "Extended" nature of the architecture allows the vector field of this ODE to be explicitly dependent on the spatial coordinates $x$.

Computational Efficiency

To optimize training, the authors implemented Constant Path Sampling. Rather than sampling random $(t, x)$ points, the model follows a single spatial point through its temporal evolution. This allows the model to recycle spatial information and utilize the adjoint method for gradient calculations, which maintains a constant memory cost regardless of the number of time steps.

Furthermore, for problems involving time-varying domains (moving boundaries), the framework utilizes a "boundary anchoring" technique. This method splits constant paths into sub-paths that remain within the valid domain, resetting the ODE at entry and exit points to ensure the solution remains constrained by boundary conditions.

Experimental Evaluation

The framework was evaluated against several synthetic benchmarks designed to test dimensionality and geometric complexity:

  • 5D Diffusion-Reaction Equation: Tested on hypercubes and irregular hyperspheres.
  • Scalability Benchmarks: Problems ranging from 4 to 64 dimensions ($d \in \{4, 8, 16, 32, 64\}$).
  • Hourglass Domain: A 1D problem featuring a time-varying spatial domain to test moving boundary handling.

Key Results

Massive Speedup and Efficiency

In a 5D benchmark, XNODE-WAN reached the target error in just 74 seconds, whereas the standard WAN required 5,865 seconds—representing a 99.8% reduction in training time. This efficiency is further reflected in iteration counts; XNODE-WAN converged in 211 iterations, compared to over 15,000 for the baseline.

Linear Scalability

While the computational cost of standard WANs increases dramatically with dimensionality, XNODE-WAN exhibits roughly linear scaling. The model successfully solved a 64-dimensional PDE in under 100 seconds while maintaining a relative error of approximately 1-2%.

Robustness

Sensitivity analysis indicates that XNODE-WAN is significantly more robust to hyperparameter selection. It maintains stable performance across a wide range of learning rates ($10^{-5}$ to $10^{-2}$), whereas standard WANs often require meticulous tuning to achieve convergence.

Conclusion

XNODE-WAN represents a significant advancement in the numerical solution of high-dimensional PDEs. By integrating the temporal continuity of Neural ODEs with the flexibility of adversarial training, the framework overcomes the efficiency bottlenecks of previous mesh-free methods. Future work may extend this approach to hyperbolic PDEs and more complex, real-world engineering simulations with highly irregular, shifting domains.

Summary updated

BibTeX
@article{ni2022towards,
  author = {Hao Ni and Paul Valsecchi Oliva and Yue Wu and Cuiyu He and Hao Ni},
  title = {{Towards fast weak adversarial training to solve high dimensional parabolic partial differential equations using XNODE-WAN}},
  journal = {Journal of Computational Physics},
  year = {2022},
  doi = {10.1016/j.jcp.2022.111233},
}
Figure