Abstract
Solving high-dimensional parabolic partial differential equations (PDEs) has long been hindered by the "curse of dimensionality." Traditional mesh-based methods scale exponentially with spatial variables, while early deep learning approaches often fail to distinguish the unique role of time in physical systems. This paper introduces XNODE-WAN, a hybrid framework that combines Weak Adversarial Networks (WAN) with Extended Neural Ordinary Differential Equations (XNODEs). By treating the PDE solution as an ODE evolving in time for fixed spatial coordinates, XNODE-WAN achieves up to a 99.8% reduction in training time compared to state-of-the-art methods and scales linearly up to 64 dimensions.
Introduction
Parabolic PDEs are essential for modeling phenomena such as heat conduction, image denoising, and derivative pricing in finance. Despite their importance, solving them in high-dimensional spaces remains a significant computational challenge. While mesh-free deep learning approaches like the Weak Adversarial Network (WAN) have made strides, they typically treat time ($t$) and space ($x$) as identical input coordinates within a standard Multi-Layer Perceptron (MLP). This approach ignores the inherent temporal flow of parabolic systems, leading to slow convergence and high data requirements.
Methodology
The XNODE-WAN Architecture
XNODE-WAN addresses the limitations of standard WANs by introducing a specialized architecture that incorporates physical inductive biases. The framework is built on two primary components:
- Adversarial Training (WAN): The system employs a minimax game between a primal network (the solution approximator) and an adversarial network (the test function). The adversarial network identifies regions where the primal solution fails to satisfy the weak form of the PDE.
- Extended Neural ODEs (XNODE): Instead of an MLP, the primal network uses Neural ODEs. For any fixed spatial point $x$, the solution is modeled as an ODE evolving over time. The "Extended" nature of the architecture allows the vector field of this ODE to be explicitly dependent on the spatial coordinates $x$.
Computational Efficiency
To optimize training, the authors implemented Constant Path Sampling. Rather than sampling random $(t, x)$ points, the model follows a single spatial point through its temporal evolution. This allows the model to recycle spatial information and utilize the adjoint method for gradient calculations, which maintains a constant memory cost regardless of the number of time steps.
Furthermore, for problems involving time-varying domains (moving boundaries), the framework utilizes a "boundary anchoring" technique. This method splits constant paths into sub-paths that remain within the valid domain, resetting the ODE at entry and exit points to ensure the solution remains constrained by boundary conditions.
Experimental Evaluation
The framework was evaluated against several synthetic benchmarks designed to test dimensionality and geometric complexity:
- 5D Diffusion-Reaction Equation: Tested on hypercubes and irregular hyperspheres.
- Scalability Benchmarks: Problems ranging from 4 to 64 dimensions ($d \in \{4, 8, 16, 32, 64\}$).
- Hourglass Domain: A 1D problem featuring a time-varying spatial domain to test moving boundary handling.
Key Results
Massive Speedup and Efficiency
In a 5D benchmark, XNODE-WAN reached the target error in just 74 seconds, whereas the standard WAN required 5,865 seconds—representing a 99.8% reduction in training time. This efficiency is further reflected in iteration counts; XNODE-WAN converged in 211 iterations, compared to over 15,000 for the baseline.
Linear Scalability
While the computational cost of standard WANs increases dramatically with dimensionality, XNODE-WAN exhibits roughly linear scaling. The model successfully solved a 64-dimensional PDE in under 100 seconds while maintaining a relative error of approximately 1-2%.
Robustness
Sensitivity analysis indicates that XNODE-WAN is significantly more robust to hyperparameter selection. It maintains stable performance across a wide range of learning rates ($10^{-5}$ to $10^{-2}$), whereas standard WANs often require meticulous tuning to achieve convergence.
Conclusion
XNODE-WAN represents a significant advancement in the numerical solution of high-dimensional PDEs. By integrating the temporal continuity of Neural ODEs with the flexibility of adversarial training, the framework overcomes the efficiency bottlenecks of previous mesh-free methods. Future work may extend this approach to hyperbolic PDEs and more complex, real-world engineering simulations with highly irregular, shifting domains.
Summary updated