Learning to predict the cosmological structure formation

Siyu He, Yin Li, Yu Feng, Shirley Ho, Siamak Ravanbakhsh, Wei Chen, Barnabás Póczos

TL;DR

Simulating the evolution of the Universe is computationally daunting, often requiring supercomputers to model gravity across billions of particles over billions of years. This paper introduces the Deep Density Displacement Model (D³M), a 3D neural network that predicts cosmological structure formation significantly faster and more accurately than traditional fast analytical approximations. Remarkably, the model can extrapolate to cosmological parameters it was never trained on, offering a powerful, highly generalizable new tool for astrophysicists.

What Problem Does This Paper Address?

To understand the Universe, astrophysicists compare telescope observations of the "Cosmic Web" (the large-scale structure of the universe) with theoretical models. To do this, they rely on N-body simulations—computational models that simulate the gravitational evolution of billions of particles.

However, full N-body simulations are incredibly computationally expensive. To get around this, researchers often use fast analytical approximations based on perturbation theory, such as the Zel'dovich Approximation (ZA) or Second-order Lagrangian Perturbation Theory (2LPT). While fast, these approximations fail in the "non-linear regime"—areas where matter densely clusters into galaxies and filaments. The authors tackle this gap by asking: can we use deep learning to map a fast, simple linear approximation (ZA) directly to a highly accurate, non-linear N-body simulation?

Methods

The authors built a 3D Convolutional Neural Network based on the U-Net architecture, dubbed D³M.

  • Inputs and Outputs: The model takes the fast, linear ZA displacement field as its input and outputs a highly accurate, non-linear displacement field (approximating a full N-body simulation).
  • Displacement vs. Density: A crucial design choice was training the model on displacement fields (vectors showing how far particles moved from their origin) rather than density fields. In highly clustered regions, different displacement fields can result in the exact same density field. By using displacements, the authors eliminate this ambiguity and give the network a clear, bijective mapping to learn from.
  • Periodic Padding: Because the simulated universe is modeled as a box with periodic boundary conditions (a torus geometry where the left edge connects to the right edge), standard CNN padding (like zero or reflective padding) breaks physical symmetries. The authors implemented custom periodic padding to ensure the neural network respects the physics of the simulation boundaries.

Datasets

  • Training/Validation/Test Data: The authors generated 10,000 pairs of simulations. The input was generated using the Zel'dovich Approximation (ZA), and the "ground truth" target was generated using FastPM (a highly accurate approximate N-body solver).
  • Specs: Each simulation contained 32³ particles in a volume of 128 $h^{-1}$Mpc (about 600 million light-years across). The data was split into 80% training, 10% validation, and 10% testing.
  • Baselines: For benchmarking, they generated 1,000 simulations using 2LPT, the current standard for fast, approximate cosmological simulations.

Key Insights and Ideas

  • Zero-Shot Generalization to New Physics: The absolute biggest "aha moment" of the paper is that D³M generalizes to universes it has never seen. Trained on a single set of cosmological parameters (specific values for matter density $\Omega_m$ and primordial amplitude $A_s$), the model accurately predicted structure formation for universes with drastically different parameters. This proves the network actually learned the underlying gravitational dynamics, rather than just memorizing a specific dataset.
  • Physics-Informed Architecture Matters: The decision to predict particle displacement rather than raw density, combined with periodic boundary padding, demonstrates that injecting domain knowledge into standard ML architectures (like U-Net) is critical for scientific machine learning.
  • Bridging the Linear/Non-Linear Gap: D³M successfully learns the deeply non-linear mappings that traditional perturbation theories struggle with, effectively bridging the gap between fast linear approximations and slow, exact N-body simulations.

Key Results

D³M significantly outperformed the traditional 2LPT baseline across multiple metrics:

  • Point-wise Error: D³M achieved an average relative displacement error of just 2.8%, compared to 9.3% for 2LPT.
  • 2-Point Correlation (Power Spectrum): This metric measures the scale of clustering. D³M's predictions differed from the ground truth by a mere 0.4% at intermediate scales, whereas 2LPT deviated significantly.
  • 3-Point Correlation (3PCF): This measures non-Gaussian structure formation (the complex shapes of the cosmic web). D³M achieved a relative residual of just 0.79%, an order of magnitude better than 2LPT's 7.82%.
  • Parameter Extrapolation: When tested on a universe with much higher matter density ($\Omega_m = 0.5$) than its training set ($\Omega_m = 0.3089$), D³M maintained a low error of 3.8%, crushing 2LPT's 15.2% error.

Limitations and Future Work

  • Approximate Ground Truth: The model was trained using FastPM, which is itself an approximate N-body solver. The authors note that replacing FastPM with exact, full N-body simulations in the training data should further improve the model's absolute accuracy.
  • Resolution Limits: The current model operates on a relatively coarse grid (32³ voxels) due to computational constraints. Because the model's complexity scales linearly with the number of voxels, future work will focus on scaling the architecture to handle higher-resolution simulations.

Summary updated

Abstract

Matter evolved under the influence of gravity from minuscule density fluctuations. Nonperturbative structure formed hierarchically over all scales and developed non-Gaussian features in the Universe, known as the cosmic web. To fully understand the structure formation of the Universe is one of the holy grails of modern astrophysics. Astrophysicists survey large volumes of the Universe and use a large ensemble of computer simulations to compare with the observed data to extract the full information of our own Universe. However, to evolve billions of particles over billions of years, even with the simplest physics, is a daunting task. We build a deep neural network, the Deep Density Displacement Model ([Formula: see text]), which learns from a set of prerun numerical simulations, to predict the nonlinear large-scale structure of the Universe with the Zel'dovich Approximation (ZA), an analytical approximation based on perturbation theory, as the input. Our extensive analysis demonstrates that [Formula: see text] outperforms the second-order perturbation theory (2LPT), the commonly used fast-approximate simulation method, in predicting cosmic structure in the nonlinear regime. We also show that [Formula: see text] is able to accurately extrapolate far beyond its training data and predict structure formation for significantly different cosmological parameters. Our study proves that deep learning is a practical and accurate alternative to approximate 3D simulations of the gravitational structure formation of the Universe.
BibTeX
@inproceedings{he2019learning,
  author = {Siyu He and Yin Li and Yu Feng and Shirley Ho and Siamak Ravanbakhsh and Wei Chen and Barnabás Póczos},
  title = {{Learning to predict the cosmological structure formation}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2019},
  doi = {10.1073/pnas.1821458116},
}

Loading related content...

Figure