Abstract
Solving high-dimensional partial differential equations (PDEs) is a fundamental challenge in scientific computing due to the "curse of dimensionality," where computational costs grow exponentially with the number of dimensions. This paper establishes a rigorous theoretical and numerical framework for solving steady-state diffusion-reaction equations in high dimensions using Physics-Informed Neural Networks (PINNs) and Compressive Fourier Collocation (CFC). We provide a "practical existence theorem" demonstrating that PINNs can achieve accurate solutions with sample complexity scaling only logarithmically or linearly with dimension. Furthermore, we introduce an adaptive CFC method that achieves near-machine precision for sparse solutions. Numerical experiments in up to 30 dimensions validate the efficiency and robustness of these approaches.
Introduction
The "curse of dimensionality," a term coined by Richard Bellman, describes the exponential explosion of computational requirements as the dimensionality ($d$) of a problem increases. In the context of Partial Differential Equations (PDEs), traditional grid-based methods become intractable for $d \gg 3$. While Deep Learning (DL) has emerged as a powerful tool for high-dimensional problems, its application to PDEs often lacks a solid theoretical foundation regarding sample complexity and trainability.
This work, published by Simone Brugiapaglia and colleagues, bridges the gap between empirical DL success and mathematical sparse approximation theory. By focusing on steady-state diffusion-reaction equations on a $d$-dimensional torus, the authors demonstrate that high-dimensional PDEs can be solved efficiently if the solution exhibits sparsity in the Fourier domain.
Methodology
Periodic Physics-Informed Neural Networks (PINNs)
To handle periodic boundary conditions naturally, the authors employ a specialized PINN architecture. This network incorporates a "periodic layer" that transforms input coordinates using sine and cosine functions before passing them to subsequent hidden layers.
The network is trained by minimizing the PDE residual at a set of random collocation points. The use of periodic layers ensures that the boundary conditions are satisfied by construction, allowing the optimization process to focus entirely on the interior physics.
Compressive Fourier Collocation (CFC)
In addition to PINNs, the paper explores Compressive Fourier Collocation (CFC), a non-DL approach rooted in Compressive Sensing. CFC assumes the solution is a sparse linear combination of Fourier basis functions. To navigate the massive search space of Fourier indices in high dimensions, the authors propose Adaptive Lower OMP, a greedy algorithm that iteratively expands the set of basis functions while maintaining a "lower set" structure.
Theoretical Framework: Practical Existence
The core contribution is a "practical existence theorem." Unlike standard universal approximation theorems, this theory accounts for sample complexity—the number of training points ($m$) required to achieve a target accuracy.
The authors prove that a class of Deep Neural Networks using Rectified Power Units (RePU) can emulate products of sines and cosines. This allows them to show that for a solution with sparsity $s$, the required sample size scales as $O(s \cdot \text{polylog}(s, d))$. This result is significant because it replaces exponential scaling with a mild logarithmic or linear dependence on the physical dimension $d$.
Numerical Evaluation
The methodologies were tested on three synthetic benchmark problems in dimensions ranging from $d=6$ to $d=30$, including highly sparse, anisotropic, and full-dimensional smooth solutions.
Key Findings
- Robustness vs. Precision: PINNs proved highly robust, consistently achieving errors between $10^{-2}$ and $10^{-3}$ across all test cases. In contrast, the adaptive CFC method achieved extraordinary precision (up to $10^{-10}$) for truly sparse solutions, effectively reaching machine precision where PINNs typically plateau.
- Scaling Success: Both methods successfully solved PDEs in 30 dimensions. Empirical results confirmed that the sample size required to maintain a target error grew linearly, rather than exponentially, as the dimension doubled from 10 to 20.
Discussion and Future Work
While the theoretical framework provides a strong foundation, an "activation gap" remains: the theory utilizes RePU activations for exact Fourier emulation, whereas numerical success is often achieved with tanh or other smooth activations. Additionally, the theory assumes the existence of a global minimizer, which may not always be reached by standard optimization algorithms like Adam or SGD.
Future research will focus on extending these methods to time-dependent PDEs and non-periodic domains. Handling Dirichlet or Neumann boundary conditions will require more sophisticated analysis of the underlying Riesz systems to ensure convergence in high-dimensional settings.
Summary updated