Abstract
Solving large-scale linear systems arising from the discretization of partial differential equations (PDEs) remains a significant computational challenge. This paper introduces a new class of neural solvers that leverage meta-networks to predict optimal weights for long-step Richardson iterations. By combining these learned weights with Nesterov momentum and multilevel frameworks, the proposed methods achieve up to a 20x speedup over standard neural solvers. Our approach demonstrates exceptional robustness in solving complex PDEs, such as anisotropic diffusion and high-wavenumber Helmholtz equations, while maintaining stability across varying grid scales.
Introduction
The primary bottleneck in scientific computing is often the solution of large-scale linear systems. Traditional iterative methods, such as Richardson or Jacobi, are straightforward to implement but suffer from slow convergence rates. Accelerating these methods typically requires precise knowledge of the system's spectral properties (e.g., eigenvalue bounds), which is computationally expensive to obtain. Furthermore, problems involving anisotropy or high-frequency oscillations often require hand-tuned parameters that fail to generalize across different physical constants or grid resolutions. This work proposes a hierarchical neural approach to automate parameter selection and accelerate convergence through momentum-based techniques.
Methodology
The proposed framework builds upon the Richardson(m) iteration by integrating deep learning and classical optimization theory.
Richardson(m) Neural Solver and Meta-Networks
Instead of a single fixed weight, the Richardson(m) solver utilizes a sequence of $m$ weights. We employ a "Meta-network"—a simple fully connected network—to map PDE physical parameters ($\mu$) directly to the optimal sequence of weights ($\omega$). This eliminates the need for manual eigenvalue analysis.
Momentum and NAG-Extension
To further accelerate convergence, we incorporate Nesterov Accelerated Gradient (NAG) techniques. We introduce a "NAG-extension" that provides enhanced stability and faster convergence compared to standard NAG by adding a correction factor to the momentum term. The relationship between these optimization methods and their iterative counterparts is summarized in the table below.
Multilevel Integration
To ensure mesh independence, the neural Richardson solver is integrated into advanced multilevel architectures, such as the Fourier Neural Solver (FNS) for anisotropic equations and the Wave-ADR Neural Solver (WANS) for the Helmholtz equation. This hybrid approach leverages the spectral bias of neural networks to learn low-frequency patterns while using Richardson iterations to smooth high-frequency errors.
Experimental Results
The methods were evaluated on 2D anisotropic diffusion and Helmholtz equations across grid scales ranging from $32 \times 32$ up to $4096 \times 4096$.
- Performance Gains: The
NAGex-Richardson(m)-NSwith SSOR preconditioning achieved a 20x speedup in iteration counts compared to the baselineRichardson(m)-NS. - Mesh Independence: While single-level methods saw iteration counts explode with grid size (e.g., from 27 to 977 iterations), the multilevel Richardson(10)-FNS remained stable, requiring only 83 iterations at $N=512$.
- Comparison to Classical Methods: The neural solvers matched or outperformed the classical Chebyshev semi-iterative method without requiring explicit eigenvalue computation.
Conclusion
This work demonstrates that meta-learning can effectively bridge the gap between deep learning and classical numerical analysis. By learning the functional relationship between a PDE's parameters and its optimal numerical weights, we create solvers that are both fast and robust. Future research will focus on extending these meta-networks to 3D systems and more complex advection-diffusion-reaction equations.
Summary updated