Abstract
Swift is a next-generation, open-source simulation framework designed to model the universe at unprecedented scales. By replacing traditional "tree-walk" algorithms with a "hydrodynamics-first" task-based architecture, it achieves massive speedups—over 7x in some cases—and scales efficiently to hundreds of billions of particles on modern supercomputers. This paper details the architectural innovations that allow Swift to exploit the full power of modern hardware.
Introduction: The Scaling Bottleneck
For decades, astrophysical simulations have relied on codes like Gadget to model galaxy formation and large-scale structure. However, these legacy codes were designed for an era of fewer, faster CPU cores. Modern supercomputers gain power through massive parallelism, involving thousands of cores and complex memory hierarchies.
The primary bottleneck in traditional codes is neighbor-finding. To calculate the interactions between particles, the code must identify all nearby neighbors. Most codes use a "tree-walk" approach, which involves non-linear memory access. On modern hardware, this leads to frequent "cache misses" and prevents the use of Single Instruction, Multiple Data (SIMD) vector instructions. Swift was built from scratch to overcome these architectural inefficiencies.
Methods: A Task-Based Architecture
Swift introduces several radical departures from standard simulation techniques to optimize performance on modern hardware.
Task-Based Parallelism
Instead of a rigid sequence of operations, Swift breaks the workload into a Directed Acyclic Graph (DAG) of individual tasks. A specialized scheduler, QuickSched, assigns these tasks to available CPU cores dynamically. This ensures high CPU utilization by preventing cores from sitting idle while waiting for global synchronization points.
Hydrodynamics-First Neighbor Finding
Instead of walking a tree, Swift utilizes a recursive grid of cells. For Smoothed Particle Hydrodynamics (SPH), the code only searches for neighbors within the same or adjacent cells.
As shown in the diagram, particles are organized into cells of size $H$. This structure allows for linear memory access and enables the code to exploit modern CPU vector units (AVX/AVX-512), significantly boosting throughput.
Gravity and Communication
For gravitational calculations, Swift employs the Fast Multipole Method (FMM), which offers $O(N)$ complexity and high accuracy by treating distant groups of particles as multipoles. Furthermore, Swift treats inter-node MPI communication as just another task in the graph, allowing it to "hide" network latency by performing local calculations simultaneously.
Key Results and Validation
Swift has been validated against established benchmarks and used for some of the largest cosmological simulations to date, such as the FLAMINGO project, which involved up to 300 billion particles.
- Performance Gains: In head-to-head comparisons running the EAGLE galaxy formation model, Swift was more than 7 times faster than the previous Gadget-based implementation.
- Scaling Efficiency: The code demonstrates near-perfect weak scaling, maintaining constant runtimes as problem size and compute resources are scaled proportionally.
- Vectorization: The use of SIMD instructions provided a 1.9x to 3.1x speedup on gravity calculations alone.
Conclusion and Future Work
Swift represents a significant leap in simulation technology, prioritizing cache locality and task-based load balancing. While currently optimized for CPUs, future development aims to integrate GPU support and expand the physics modules to include magnetohydrodynamics (MHD) and radiative transfer. The transition to task-based systems requires a shift in developer mindset but is essential for leveraging the next generation of exascale supercomputers.
Summary updated