Abstract
Simulating fluid turbulence at the viscous limit remains a foundational challenge in computational physics due to the extreme multiscale nature of energy dissipation. While deep learning models like Vision Transformers (ViTs) offer promise, they typically rely on patch-based tokenization that blurs critical small-scale eddies. We present a novel framework that achieves pixel-level resolution by combining a physics-inspired hierarchical architecture, the Hierarchical Turbulence Transformer (Matey), with a high-performance sequence parallelism algorithm, RingX. Our approach achieved a sustained 1.1 EFLOPS on the Frontier supercomputer with 94% scaling efficiency. By resolving features down to the Kolmogorov scale, this work marks a significant milestone in applying exascale AI to high-fidelity fluid dynamics.
Introduction: The Multiscale Challenge
Fluid turbulence is characterized by a cascade of energy from large-scale swirling flows down to trillions of tiny, energy-dissipating eddies. Capturing this "viscous limit" requires massive spatial resolution, often involving billions or trillions of grid points. In the context of deep learning, this presents a significant computational bottleneck. Standard Vision Transformers (ViTs) manage memory by grouping pixels into large patches, but this "patching" effectively acts as a low-pass filter, removing the high-frequency physical information necessary for accurate turbulence modeling.
Processing every pixel as an individual token would result in sequence lengths that exceed the memory and compute capacity of even the most advanced hardware, given the $O(S^2)$ complexity of standard attention mechanisms. To address this, we introduce a dual-pronged innovation: a hierarchical model structure that mirrors physical scales and an HPC-optimized attention algorithm.
The Hierarchical Turbulence Transformer (Matey)
The "Matey" architecture is designed to mimic the classical hierarchical approach to fluid dynamics, which categorizes flows into Reynolds-Averaged Navier-Stokes (RANS), Large Eddy Simulation (LES), and Direct Numerical Simulation (DNS) regimes.
As shown in the figure above, the model splits the input into three fidelity modes:
- Low-fidelity: Captures global, long-range physics at a coarse resolution.
- Mid-fidelity: Focuses on intermediate scales within specific subdomains.
- High-fidelity: Resolves local, fine-scale physics at the individual pixel level.
By structuring the Transformer in this way, we reduce the effective sequence length from billions of tokens to a manageable few million, while ensuring that the smallest physical features—the dissipative eddies—are explicitly resolved rather than blurred.
RingX: Scaling Attention for HPC
While the hierarchical structure reduces the token count, exascale performance still requires highly efficient parallelization. Standard "Ring Attention" utilizes point-to-point communication to pass data between GPUs in a circular topology. However, this approach often fails to fully exploit the high-bandwidth collective communication primitives available on modern supercomputer interconnects.
We developed RingX, a sequence parallelism algorithm optimized for HPC environments like the Frontier supercomputer. RingX replaces simple point-to-point transfers with optimized collective communications, such as Broadcast and Allreduce, and improves the overlap between computation and communication.
Empirical results demonstrate that RingX provides up to a 2.5x speedup over standard Ring Attention. This efficiency was critical in achieving a sustained performance of 1.1 EFLOPS across 32,768 AMD GPUs, maintaining a 94% scaling efficiency.
Experimental Validation and Physical Accuracy
The framework was evaluated using two gold-standard turbulence benchmarks: Forced Homogeneous Isotropic Turbulence (HIT) and Stably Stratified Turbulence (SST).
In the HIT tests, which utilize $1024^3$ grid points from the Johns Hopkins Turbulence Database, the model's predictions for the energy spectrum and enstrophy (a measure of local rotation) almost perfectly matched ground-truth DNS data. Crucially, the model successfully captured the "dissipative range"—the scale at which turbulence dies out—a feat that previous AI models have struggled to achieve.
In SST cases, the model demonstrated an ability to generalize to complex interactions, such as buoyancy and the transition from laminar to chaotic flow. While there was a slight under-prediction of total energy in the initial time steps of stratified flows—likely due to the complexity of buoyancy flux—the overall physical fidelity remained remarkably high.
Conclusion and Future Directions
The success of the Matey architecture and RingX algorithm suggests a "Physics-AI Symmetry": the most effective AI models for scientific discovery are those that mirror the underlying structure of physical laws. By redesigning context parallelism for HPC hardware, we have shown that it is possible to resolve the smallest scales of turbulence at exascale.
Future work will focus on "in-situ" learning, where the AI model is trained concurrently with a physics simulation. This would eliminate the need for massive intermediate datasets and allow for even higher-resolution modeling. Furthermore, the hierarchical approach developed here holds potential for other multiscale scientific domains, including climate modeling and genomics.
Summary updated