Abstract
This paper introduces a high-performance out-of-core (OOC) Cholesky factorization algorithm optimized for large-scale matrices that exceed GPU memory capacity. By utilizing a static task scheduler and the high-bandwidth NVLink-C2C interconnect on NVIDIA Grace Hopper systems, the proposed method achieves significant performance gains. A key contribution is a novel four-precision (FP64 to FP8) mixed-precision strategy that provides a 3X speedup over standard double-precision implementations while maintaining the accuracy required for geospatial statistical applications.
Introduction
In fields such as geospatial statistics, researchers frequently perform Cholesky factorization on enormous matrices to model spatial data. However, two major bottlenecks hinder performance: the "Memory Wall" and the "Interconnect Bottleneck." Modern GPUs possess immense computational power but limited onboard VRAM. When a matrix exceeds this capacity, it must be stored in CPU memory and swapped in and out—a process known as "Out-of-Core" (OOC) computing. Traditional PCIe links are often too slow to keep the GPU fed with data, leading to processor idling. This gap is further widened by mixed-precision (MxP) computing, which accelerates GPU processing even further beyond the data transfer rate.
Methods
The authors introduce several key innovations to bridge the gap between compute speed and data movement:
- Left-Looking Tile-Based Algorithm: Unlike "right-looking" variants that update the entire remaining matrix at once, the "left-looking" approach is "lazy." It prioritizes finishing specific tiles along the critical path, promoting better data reuse and reducing the frequency of host-device transfers.
- Static Task Scheduling: Instead of a dynamic runtime, the authors use a deterministic static scheduler. Because the execution order is known in advance, the system can aggressively overlap asynchronous data transfers (CPU-to-GPU and GPU-to-CPU) with computations, effectively hiding the latency of moving data.
- NVLink-C2C Integration: The work targets the NVIDIA GH200 Grace Hopper Superchip, utilizing its 900 GB/s coherent interconnect to move data between the Grace CPU and Hopper GPU significantly faster than PCIe 4.0 (32 GB/s).
Four-Precision Mixed-Precision (MxP) Strategy
A major highlight of this research is the dynamic assignment of four precisions (FP64, FP32, FP16, or FP8) to different tiles of the matrix. The algorithm uses a heuristic based on the Frobenius norm to identify tiles that contribute less to the final result's significance. These tiles are processed in lower precision, while critical tiles remain in higher precision to maintain numerical stability.
In spatial statistics, tiles representing distant points (weak correlation) can often be stored in FP8. As correlation increases, the algorithm automatically shifts toward higher precision (FP32/FP64). This "lazy" strategy minimizes the "working set" of data needed at any given time, which is optimal for OOC scenarios.
Key Results
The researchers evaluated their implementation using synthetic datasets for large-scale geospatial statistical applications, with matrix sizes reaching up to 200,000 x 200,000—well beyond the 80GB memory capacity of a single GH200 GPU.
- Performance: On a single GH200, the proposed FP64 implementation outperformed NVIDIA’s cuSOLVER by 20%.
- Mixed-Precision Speedup: Enabling the four-precision MxP mode provided a 3X performance boost over the FP64-only version, reaching peak throughputs of 136.1 TFlop/s.
- Scalability: The algorithm scales almost linearly when moving from one to four GH200 superchips.
- Accuracy: The MxP implementation maintained "application-worthy" accuracy, as measured via Kullback-Leibler divergence.
Conclusion and Future Work
The transition from PCIe to NVLink-C2C fundamentally changes OOC algorithm design, allowing CPU memory to be treated almost as an extension of GPU VRAM. While the current heuristic for switching precisions is tuned for geospatial statistics, future work involves developing generalized heuristics for other domains like fluid dynamics. Additionally, the authors suggest extending this static scheduling and OOC framework to other essential linear algebra operations, such as LU and QR factorizations.
Summary updated