Abstract
Understanding long-term material evolution, such as the embrittlement of nuclear reactor steel, requires simulating rare-event dynamics over vast spatial and temporal scales. Traditional Kinetic Monte Carlo (KMC) methods are computationally prohibitive for billion-atom systems due to their passive sampling nature. We present SwarmThinkers, a reinforcement learning (RL) framework that transforms atoms into autonomous agents. By combining a decentralized policy with a novel reweighting mechanism, SwarmThinkers maintains physical rigor while achieving a 3,185x speedup over state-of-the-art KMC tools. We demonstrate the first physically consistent simulation of 54 billion atoms on a single GPU, a feat previously requiring supercomputer clusters.
Introduction
In materials science, the "rare-event" problem is a significant bottleneck. Processes like copper clustering in iron alloys occur over decades but are driven by atomic hops happening on picosecond scales. Kinetic Monte Carlo (KMC) is the standard for simulating these dynamics, but it suffers from "passive" sampling. Most computational effort is wasted on reversible fluctuations—local "noise"—rather than the transitions that drive long-term structural evolution.
Methodology: Atoms as Intelligent Agents
SwarmThinkers reimagines the simulation space as a multi-agent reinforcement learning environment where atoms are decision-making entities.
Decentralized Policy and Centralized Critic
Each diffusing particle acts as an agent, observing its local neighborhood (up to the second nearest neighbor). A shared neural network policy proposes movement directions. During training, a global "critic" evaluates the system state, rewarding moves that lead to thermodynamic relaxation and lower total system energy. This Centralized Training, Decentralized Execution (CTDE) paradigm allows the model to be trained on small lattices (e.g., 10x10x10) and deployed on massive systems without retraining, as agents rely solely on local observations.
Maintaining Physical Rigor
To ensure the simulation obeys the laws of thermodynamics, SwarmThinkers does not follow the RL policy blindly. It employs a Reweighting Mechanism that fuses learned agent preferences with classical Arrhenius transition rates. Furthermore, trajectory-level Importance Sampling is applied to correct any bias introduced by the RL policy, ensuring that macroscopic observables remain statistically identical to traditional physical simulations.
Experimental Results
The framework was validated using the Fe–Cu (Iron-Copper) alloy precipitation benchmark, comparing results against the OpenKMC baseline and historical experimental data.
Physical Fidelity and Accuracy
The "advancement factor" ($ζ$), which measures material evolution over time, shows that SwarmThinkers perfectly matches both simulation baselines and experimental measurements across a wide temperature range (663K to 773K). This confirms that the RL-guided transitions do not sacrifice physical accuracy for speed.
Unprecedented Scalability and Performance
SwarmThinkers achieved a simulation of a 2000 × 3000 × 9000 lattice containing 54 billion atoms on a single NVIDIA A100 GPU. Key performance metrics include:
- Massive Speedup: An average of 3,185x over OpenKMC, peaking at 4,963x.
- Extreme Efficiency: 485x less memory usage than traditional methods. The 54-billion-atom case required less than 60GB of VRAM, whereas traditional methods would require nearly 40 Terabytes of memory across a supercomputer cluster.
Conclusion
SwarmThinkers represents a paradigm shift from passive atomic sampling to active, intelligent simulation. By bridging reinforcement learning with thermodynamic principles, it enables massive-scale materials discovery on commodity hardware. Future work will extend this framework to complex multi-element alloys and a broader range of rare-event types, such as dislocation movement and grain boundary migration.
Summary updated