Learning to Predict the Cosmological Structure Formation

Siyu He, Yin Li, Yu Feng, Shirley Ho, Siamak Ravanbakhsh, Wei Chen, Barnabás Póczos

Abstract: Matter evolved under influence of gravity from minuscule density fluctuations. Non-perturbative structure formed hierarchically over all scales, and developed non-Gaussian features in the Universe, known as the Cosmic Web. To fully understand the structure formation of the Universe is one of the holy grails of modern astrophysics. Astrophysicists survey large volumes of the Universe and employ a large ensemble of computer simulations to compare with the observed data in order to extract the full information of our own Universe. However, to evolve trillions of galaxies over billions of years even with the simplest physics is a daunting task. We build a deep neural network, the Deep Density Displacement Model (hereafter D³M), to predict the non-linear structure formation of the Universe from simple linear perturbation theory. Our extensive analysis demonstrates that D³M outperforms the second order perturbation theory (hereafter 2LPT), the commonly used fast approximate simulation method, in point-wise comparison, 2-point correlation, and 3-point correlation. We also show that D³M is able to accurately extrapolate far beyond its training data, and predict structure formation for significantly different cosmological parameters. Our study proves, for the first time, that deep learning is a practical and accurate alternative to approximate simulations of the gravitational structure formation of the Universe.

Introduction

To understand the fundamental nature of our Universe, astrophysicists compare telescope observations of the "Cosmic Web"—the vast, large-scale structure of matter—with theoretical models. This comparison relies heavily on N-body simulations, which are computational models that simulate the gravitational evolution of billions of particles over cosmic time.

However, full N-body simulations are incredibly computationally expensive, often requiring supercomputers. To mitigate this, researchers frequently use fast analytical approximations based on perturbation theory, such as the Zel'dovich Approximation (ZA) or Second-order Lagrangian Perturbation Theory (2LPT). While these methods are fast, they struggle in the "non-linear regime," where matter clusters densely into galaxies and filaments. This paper introduces the Deep Density Displacement Model (D³M), a 3D neural network designed to bridge this gap by mapping simple linear approximations directly to highly accurate non-linear simulations.

Methods and Architecture

The D³M architecture is a 3D Convolutional Neural Network based on the U-Net design. A critical innovation in this work is the choice of training data: the model is trained on displacement fields (vectors representing particle movement) rather than density fields. In highly clustered regions, different displacement fields can result in the same density field; by using displacements, the authors establish a clear, bijective mapping for the network to learn.

Visual comparison of cosmological structure predictions across different models. D³M (d) shows significantly better agreement with the FastPM ground truth (a) than the analytical baselines (b, c).

Furthermore, the model incorporates custom periodic padding. Because simulated universes are modeled as boxes with periodic boundary conditions, standard zero or reflective padding would break physical symmetries. Periodic padding ensures the neural network respects the underlying physics of the simulation boundaries.

Datasets

The authors generated 10,000 simulation pairs for training. The input was generated using the fast Zel'dovich Approximation (ZA), while the "ground truth" target was produced using FastPM, a highly accurate approximate N-body solver. Each simulation modeled 32³ particles in a volume of 128 $h^{-1}$Mpc. For benchmarking, 1,000 simulations were also generated using 2LPT, the current industry standard for fast approximations.

Key Results

D³M significantly outperformed the traditional 2LPT baseline across all primary cosmological metrics:

  • Point-wise Accuracy: D³M achieved an average relative displacement error of just 2.8%, compared to 9.3% for 2LPT.
  • 2-Point Correlation (Power Spectrum): This metric measures the scale of clustering. D³M's predictions differed from the ground truth by a mere 0.4% at intermediate scales.
  • 3-Point Correlation (3PCF): This measures complex, non-Gaussian shapes within the cosmic web. D³M achieved a relative residual of 0.79%, an order of magnitude better than 2LPT's 7.82%.
Statistical performance metrics. D³M (green) tracks the ground truth (dashed black line) much more closely than the 2LPT baseline (blue) in both 2-point and 3-point correlation analyses.

Generalization and Extrapolation

Perhaps the most significant finding is D³M's ability to generalize to "new physics." Despite being trained on a single set of cosmological parameters (specific values for matter density $\Omega_m$ and primordial amplitude $A_s$), the model accurately predicted structure formation for universes with drastically different parameters.

For example, when tested on a universe with a matter density of $\Omega_m = 0.5$ (compared to the training value of $\Omega_m \approx 0.31$), D³M maintained a low error of 3.8%, while the 2LPT baseline's error spiked to 15.2%. This suggests the network has learned the fundamental gravitational dynamics rather than simply memorizing the training distribution.

Conclusion and Future Work

D³M demonstrates that deep learning is a viable, high-speed alternative to traditional approximate simulations. While currently limited by the resolution of the training grid and the use of FastPM as a ground truth, future iterations scaling to higher resolutions and training on exact N-body simulations promise to further revolutionize cosmological modeling. By injecting domain knowledge—such as displacement mapping and periodic symmetries—into deep learning architectures, we can create tools that are both physically consistent and computationally efficient.

Summary updated

Abstract

Matter evolved under influence of gravity from minuscule density fluctuations. Non-perturbative structure formed hierarchically over all scales, and developed non-Gaussian features in the Universe, known as the Cosmic Web. To fully understand the structure formation of the Universe is one of the holy grails of modern astrophysics. Astrophysicists survey large volumes of the Universe and employ a large ensemble of computer simulations to compare with the observed data in order to extract the full information of our own Universe. However, to evolve trillions of galaxies over billions of years even with the simplest physics is a daunting task. We build a deep neural network, the Deep Density Displacement Model (hereafter D$^3$M), to predict the non-linear structure formation of the Universe from simple linear perturbation theory. Our extensive analysis, demonstrates that D$^3$M outperforms the second order perturbation theory (hereafter 2LPT), the commonly used fast approximate simulation method, in point-wise comparison, 2-point correlation, and 3-point correlation. We also show that D$^3$M is able to accurately extrapolate far beyond its training data, and predict structure formation for significantly different cosmological parameters. Our study proves, for the first time, that deep learning is a practical and accurate alternative to approximate simulations of the gravitational structure formation of the Universe.
BibTeX
@inproceedings{he2019learning,
  author = {Siyu He and Yin Li and Yu Feng and Shirley Ho and Siamak Ravanbakhsh and Wei Chen and Barnabás Póczos},
  title = {{Learning to Predict the Cosmological Structure Formation}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2019},
  doi = {10.1073/pnas.1821458116},
}
Figure