Traditional numerical methods for solving partial differential equations (PDEs), such as Finite Element Methods, rely on discretizing space into a grid or mesh. This approach works well in two or three dimensions, but it fails catastrophically as dimensionality increases—a phenomenon known as the "curse of dimensionality." In 100 dimensions, even a coarse grid with only two points per dimension would require $2^{100}$ cells, far exceeding the capacity of any computer.
Weinan E, Jiequn Han, and Arnulf Jentzen argue in Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning that a new generation of algorithms is breaking this curse by replacing grids with neural networks. Specifically, the Deep Ritz Method (DRM) shifts the problem from solving a differential equation directly to minimizing a physical energy functional.
From Equations to Energy
Most physics problems can be framed in two ways: the strong form (a differential equation that must equal zero everywhere) and the variational form (a quantity, like energy, that the system naturally seeks to minimize).
In the Deep Ritz Method, we treat a neural network as a "trial function." Instead of trying to satisfy a complex PDE at every point, we train the network to minimize an energy functional derived from the variational principle. As Rui Du and Keke Wu explain in A comprehensive study of boundary conditions when solving PDEs by DNNs, this transforms the PDE into an optimization problem. Because neural networks are mesh-free, we can evaluate this energy by sampling random points in high-dimensional space, bypassing the need for a grid entirely.
The Advantage of Lower Order Derivatives
A key intuition for why the Deep Ritz Method often outperforms residual-based methods (like the Deep Galerkin Method or standard PINNs) lies in the "order" of the math involved.
Solving a second-order PDE (like the Poisson equation) requires calculating second-order derivatives of the neural network during training. However, the variational energy functional typically only requires first-order derivatives (the Dirichlet energy). Somdatta Goswami and colleagues demonstrate in Transfer learning enhanced physics informed neural network for phase-field modeling of fracture that this reduction in derivative order leads to a smoother optimization landscape.
By requiring only first-order information, the training process becomes more stable and computationally efficient. Yuxiang Gao and Ravindra Duddu further note in Physics informed cell representations for variational formulation of multiscale problems that this "weak form" allows for simpler interpolation methods, significantly reducing the overhead of automatic differentiation.
Why It Scales: Spectral Barron Spaces
While the empirical success of these methods is clear, researchers have also sought to understand why neural networks don't succumb to the curse of dimensionality in these settings.
Jianfeng Lu and Yulong Lu provided a theoretical breakthrough in A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Partial Differential Equations. They proved that if the solution to a PDE belongs to a specific low-complexity function space called the Spectral Barron space, the error of the Deep Ritz Method decreases at a rate independent of the number of dimensions.
This suggests that many high-dimensional physical systems possess an underlying structure that neural networks are uniquely suited to capture. This theoretical foundation extends even to nonlinear problems, as shown by Tanya Marwah and Andrej Risteski in Neural Network Approximations of PDEs Beyond Linearity: A Representational Perspective, where they prove that "neurally unfolding" the optimization process allows for efficient representation of nonlinear solutions.
Beyond Physics: Generative Modeling and Sampling
The intuition of the Deep Ritz Method—solving PDEs by minimizing variational energy—is now being exported to other fields, most notably generative modeling and statistics.
In Variational Potential Flow: A Novel Probabilistic Framework for Energy-Based Generative Modelling, Junn Yong Loo and Chee-Ming Ting apply the Deep Ritz approach to the Fokker-Planck equation. Instead of using unstable MCMC sampling to train generative models, they frame the problem as a "potential flow" where the model learns an energy function that satisfies physical probability constraints.
Similarly, Artur Goldman and Denis Belomestny use neural networks to solve the Poisson equation in Theoretical guarantees for neural control variates in MCMC, using the solution to cancel out noise in Monte Carlo simulations. In both cases, the core mechanism remains the same: treat the neural network as a flexible tool for variational optimization, turning hard differential problems into solvable energy minimizations.