Ab Initio Accuracy Beyond the Cutoff: Distributed Message-Passing Equivariant Attention at Tens-of-Millions-of-Atom Scale

Allex Kan, Ji Zhang, Hao Yin, Hao Li, Owen Fu, Xunxing Chen, Lin Huang, Zion Wang, Lucas Li, Jiajun Cheng, Arthur Jiang, Junping ZhaoarXiv 2026 · arXiv

TL;DR

The authors present Tessera, a distributed runtime that enables equivariant message-passing Transformers to scale to tens of millions of atoms on hundreds of GPUs. By co-designing the algorithm and the system—specifically through space-filling curve partitioning and communication-compute overlap—they achieve ab initio levels of accuracy at a scale previously reserved for much simpler empirical models.

What Problem Does This Paper Address?

In molecular dynamics (MD), researchers face a classic "accuracy vs. scale" trade-off. First-principles methods like Density Functional Theory (DFT) are highly accurate but computationally expensive ($O(N^3)$), limiting them to a few thousand atoms. Conversely, empirical force fields can simulate millions of atoms but fail to capture complex chemistry like bond breaking or charge transfer.

Equivariant Graph Neural Networks (GNNs) offer a middle ground: they learn the accuracy of DFT at a linear cost. However, distributing these models across many GPUs is notoriously difficult. Because atoms move, the "neighbor graph" (who interacts with whom) changes every step, leading to unpredictable communication patterns and variable tensor shapes that break standard optimization techniques like CUDA Graphs. Furthermore, the "message-passing" nature of these models requires multiple rounds of data exchange per simulation step, which quickly becomes a massive communication bottleneck.

Methods

The authors introduce Tessera, a runtime built on four key pillars of algorithm-system co-design:

  1. Spatial Partitioning via Hilbert Curves: Instead of dividing space into equal volumes (which leads to unbalanced atom counts), Tessera maps 3D coordinates onto a 1D Hilbert space-filling curve. It then cuts this curve into equal-count shards. This ensures every GPU handles the exact same number of atoms, fixing tensor shapes and allowing for efficient CUDA Graph capture. Spatial decomposition using Hilbert curves to fix tensor shapes.
Spatial decomposition using Hilbert curves to fix tensor shapes.
  1. Multi-Level Neighbor Search: To find interacting atoms across devices, they use a three-level filter: a coarse axis-aligned bounding box (AABB) filter between devices, balanced query chunks within a device, and an exact search using the FAISS library. The three-level neighbor search pipeline.
The three-level neighbor search pipeline.
  1. Communication-Compute Overlap: They split the attention mechanism into "local" and "remote" branches. By launching the remote data exchange asynchronously, they can hide the communication latency behind the local computation. Overlapping remote communication with local computation.
Overlapping remote communication with local computation.
  1. Decentralized MD Engine: Most MD codes rely on a "Rank 0" coordinator to manage the simulation loop, creating a serial bottleneck. Tessera decentralizes this, letting each GPU integrate the motion of its own atoms locally. Decentralized on-GPU dynamics to remove serial bottlenecks.
Decentralized on-GPU dynamics to remove serial bottlenecks.

Datasets

The system was evaluated using several benchmarks:

  • Bulk Water: Used for weak-scaling studies, growing from 10k to 60k atoms per GPU.
  • Aqueous KCl Solution: A 10.24-million-atom salt solution used for long-term trajectory validation and energy drift measurements.
  • Norovirus Capsid: A 10.06-million-atom heterogeneous system (protein-dense regions in water) to test the limits of the equal-count partitioning strategy.

Key Insights and Ideas

  • "Equal-Count" is the Key to Replay: By forcing each GPU to own an identical number of atoms, the authors turned a dynamic problem into a static one. This allowed them to use CUDA Graphs to "replay" entire simulation intervals, bypassing the high overhead of the Python/CPU-side dispatcher.
  • Locality Limits the Peer Set: Using the Hilbert curve ensures that the number of "neighboring" GPUs a single GPU must talk to grows sublinearly (roughly $K^{0.5}$) rather than linearly with the total number of GPUs ($K$). This is crucial for scaling to 512+ GPUs.
  • Equivariance is Communication-Heavy: The paper highlights that equivariant features (vectors and tensors) are much larger than the simple scalars used in classical MD. Their "Sender-side projection" trick reduces this traffic by ~26% by projecting features into a more compact space before shipping them across the network.

Key Results

  • Massive Scale: Sustained a simulation of 30.7 million atoms on 512 NVIDIA H100 GPUs.
  • High Efficiency: Achieved 89.7% weak-scaling efficiency. As the problem and machine size grew 16x, the time per step only increased by 1.11–1.29x. Weak scaling results showing high efficiency up to 512 GPUs.

  • Speedups: Reusing the neighbor graph and communication plan cut the full-step time by 1.87x.

  • Physical Validity: In a 10 ps NVE simulation of a salt solution, the energy drift was a negligible $0.42 \mu eV/atom/ps$, and the structural properties perfectly matched a serial reference.
Weak scaling results showing high efficiency up to 512 GPUs.

Limitations and Future Work

  • Strong Scaling "Knee": The authors noted that strong scaling (keeping the problem size fixed while adding GPUs) hits a point of diminishing returns around 8k atoms per GPU, where allocator pressure and small batch sizes degrade performance.
  • Heterogeneity Penalty: While the equal-count cut balances the number of atoms, it doesn't perfectly balance the number of edges (interactions). In the Norovirus capsid test, protein-dense regions had more edges per atom, leading to a 14.5% performance penalty compared to uniform water.
  • Precision Trade-offs: Using TF32 precision provides a significant speedup but introduces some energy drift. Finding the perfect split between high-precision and low-precision operations remains an open challenge.
  • Future Directions: The authors suggest that a distributed sort and a sparser request structure are the next logical steps to reach the billion-atom scale.

Summary updated

Abstract

Molecular dynamics predicts molecular motion from atomic forces and trajectory integration. Density functional theory (DFT) provides first-principles forces, but its $O(N^3)$ cost limits simulations to thousands of atoms; empirical force fields scale to millions but miss bond breaking and charge transfer. Equivariant message-passing Transformers learn the ab initio potential at linear cost. Stacked layers propagate information beyond a single cutoff, giving four layers a receptive field up to 23Å. At this frontier, scale is limited by communication: layers exchange boundary features across devices, force backpropagation repeats the exchange, and moving atoms change neighbor graphs and tensor shapes. We present Tessera, a distributed runtime that fixes workload shapes between rebuilds. It sorts atoms along a space-filling curve into equal-count shards, ensuring identical atom counts and tensor shapes per rank. Peer sets scale as $K^{0.5}$, not $K$. A conservative neighbor skin preserves topology across updates, enabling each rebuild interval to replay as one CUDA Graph. Fused kernels, three-level neighbor search, and overlapped All-to-All complete it. On 512 GPUs, Tessera sustains 30.7M atoms at 89.7% weak-scaling efficiency (77--90% across three per-GPU loads); per-step time rises 1.11--1.29x as machine and problem grow 16x. Graph reuse cuts full-step time $1.87\times$. Across an $8\times$ device increase, neighbor-search cost grows $1.7\times$, versus $3.0\times$ for Morton order. In a live $10.24$M-atom salt solution, NVE energy drift is $0.42\,μ$eV/atom/ps over $10$ ps, and structure matches a serial reference. A 10.06M-atom Norovirus capsid sustains 100 ps of NVT at $\approx 8$ ps/day on 256 A100 GPUs. Over 200,000 steps, it has no broken bonds or anomalously close pairs and uses three compiled variants, with no graph breaks or eager fallbacks.
BibTeX
@misc{kan2026ab,
  author = {Allex Kan and Ji Zhang and Hao Yin and Hao Li and Owen Fu and Xunxing Chen and Lin Huang and Zion Wang and Lucas Li and Jiajun Cheng and Arthur Jiang and Junping Zhao and Jia Zhang},
  title = {{Ab Initio Accuracy Beyond the Cutoff: Distributed Message-Passing Equivariant Attention at Tens-of-Millions-of-Atom Scale}},
  year = {2026},
  eprint = {2611.06122},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06122},
}
Figure