Accurate Machine-learning Atmospheric Retrieval via a Neural-network Surrogate Model for Radiative Transfer

Michael D. Himes, Joseph Harrington, Adam D. Cobb, Atılım Güneş Baydin, Frank Soboczenski, Molly D. O’Beirne, Simone Zorzan, David Wright, Zacchaeus Scheffer, Shawn Domagal‐Goldman, Giada ArneyThe Planetary Science Journal 2022 · DOI

TL;DR

This paper introduces a new machine learning approach for analyzing exoplanet atmospheres. Instead of directly learning to predict atmospheric properties from spectra, they train a neural network to mimic the complex radiative transfer simulations used in traditional methods. This "surrogate model" allows for significantly faster retrievals within a standard Bayesian framework, achieving comparable accuracy to existing, slower techniques.

What Problem Does This Paper Address?

Characterizing exoplanet atmospheres is crucial for understanding planetary formation and habitability. The primary method for this is "atmospheric retrieval," which involves comparing observed spectra of exoplanets to theoretical models. These models rely on computationally intensive "radiative transfer" (RT) simulations to predict what a spectrum should look like for a given set of atmospheric parameters (like temperature, pressure, and gas abundances).

The challenge is that exoplanet observations often have low signal-to-noise ratios, necessitating a Bayesian approach. This means running thousands or even millions of RT simulations to explore the parameter space and determine the most likely atmospheric properties and their uncertainties. This process is extremely time-consuming, often taking days or weeks. While machine learning (ML) has shown promise in speeding up retrievals by learning inverse models (predicting parameters from spectra), these methods often sacrifice accuracy. This paper aims to bridge this gap by offering a fast and accurate retrieval method.

Methods

The core innovation is training a forward neural network surrogate model. Instead of learning the inverse mapping (spectrum -> parameters), they train a neural network to learn the forward mapping (parameters -> spectrum). This surrogate model acts as a fast approximation of the computationally expensive radiative transfer (RT) code.

Here's the breakdown:

  1. Data Generation (MARGE): They first generate a large dataset of synthetic exoplanet spectra by running the traditional, high-fidelity BART (Bayesian Atmospheric Radiative Transfer) code across a defined parameter space. This parameter space includes temperature-pressure profiles, abundances of key gases (H2O, CO2, CO, CH4), and planetary properties (radius, mass, semi-major axis).
  2. Surrogate Model Training (MARGE): A neural network (specifically, a convolutional neural network architecture) is trained on this generated dataset to accurately predict the spectrum given a set of atmospheric parameters. This trained network becomes the fast "RT simulator."
  3. Bayesian Retrieval (HOMER): This trained neural network surrogate is then integrated into a standard Bayesian retrieval framework. The HOMER package uses this fast surrogate model within a Markov Chain Monte Carlo (MCMC) sampler to explore the posterior probability distributions of the atmospheric parameters, given the observed spectrum. This approach preserves the accuracy of traditional Bayesian inference while leveraging the speed of the ML surrogate.

Datasets

  • Training/Validation/Testing Data: The authors generated a large synthetic dataset of exoplanet emission spectra using the BART radiative transfer code. This dataset comprised over 3.4 million spectra, split into training (~70%), validation (~20%), and testing (~10%) sets. The spectra cover a wavenumber range of 280-7100 cm⁻¹ at a resolution of 1.0 cm⁻¹.
  • Observational Data: The method was demonstrated on the emission spectrum of the exoplanet HD 189733 b, using publicly available data from the Hubble Space Telescope (Near Infrared Camera Multi-Object Spectrograph) and Spitzer Space Telescope (Infrared Spectrograph and Infrared Array Camera).

Key Insights and Ideas

  • Forward Model Surrogates for Accuracy: The key insight is that training a neural network to emulate the forward radiative transfer process, rather than directly learning the inverse retrieval, preserves the accuracy of traditional Bayesian inference. This circumvents the accuracy trade-off seen in many previous ML retrieval methods.
  • MARGE & HOMER: Modular Tools: The development of two open-source Python packages, MARGE (for training the surrogate) and HOMER (for performing the retrieval), makes this approach accessible to the wider community.
  • Speed-Accuracy Trade-off Re-evaluated: While ML methods can offer massive speedups, this paper shows that a significant speedup (~9-180x) is achievable without sacrificing the high accuracy typically associated with full RT simulations. This opens the door for more complex models (like 3D retrievals) to become computationally feasible.
  • Importance of Data Set Size: The study highlights that the size and comprehensiveness of the training dataset are critical for the performance of the surrogate model, even more so than minor architectural variations or learning rate policies.

Key Results

  • Accuracy: The retrieved atmospheric parameters and spectral fits from HOMER (using the ML surrogate) showed excellent agreement with those obtained from the traditional BART code. Bhattacharyya coefficients between the 1D marginalized posteriors were consistently high, ranging from 0.9843 to 0.9972, with a mean of 0.9925.
  • Speedup:
    • On a CPU (AMD EPYC 7402P), HOMER was approximately 9x faster per parallel chain than BART.
  • On a GPU (NVIDIA Titan Xp), HOMER was 90-180x faster per chain than BART.
  • Computational Cost: While the initial training of the MARGE model is computationally expensive, subsequent retrievals using HOMER are significantly faster. For multiple retrievals within the same parameter space, the MARGE+HOMER approach becomes more cost-effective than repeated BART retrievals.

Limitations and Future Work

  • Model Inaccuracies: The accuracy of the HOMER retrieval is inherently limited by the accuracy of the MARGE surrogate model. While minor inaccuracies did not significantly affect the conclusions for HD 189733 b at the tested resolution, this could be a concern for higher spectral resolutions or cases where physics-based retrievals are unavailable for comparison.
  • Architecture Selection: The paper emphasizes the need for careful selection of the neural network architecture to ensure accurate modeling across the desired parameter space and spectral resolution.
  • GPU vs. CPU: The speedup figures are impressive, but the authors note that the advantage over a GPU-accelerated RT code would be less pronounced. Further investigation into this comparison is suggested.
  • Benchmarking: The authors advocate for the creation of standardized datasets to benchmark future ML retrieval algorithms, allowing for easier comparison and validation across different methods.
  • Broader Applications: The MARGE and HOMER frameworks are presented as general tools that can be applied to modeling any deterministic process, not just atmospheric radiative transfer, suggesting potential applications in areas like cloud/haze formation, photochemistry, and even mapping gravitationally lensed galaxies.

Summary updated

Abstract

Abstract Atmospheric retrieval determines the properties of an atmosphere based on its measured spectrum. The low signal-to-noise ratios of exoplanet observations require a Bayesian approach to determine posterior probability distributions of each model parameter, given observed spectra. This inference is computationally expensive, as it requires many executions of a costly radiative transfer (RT) simulation for each set of sampled model parameters. Machine learning (ML) has recently been shown to provide a significant reduction in runtime for retrievals, mainly by training inverse ML models that predict parameter distributions, given observed spectra, albeit with reduced posterior accuracy. Here we present a novel approach to retrieval by training a forward ML surrogate model that predicts spectra given model parameters, providing a fast approximate RT simulation that can be used in a conventional Bayesian retrieval framework without significant loss of accuracy. We demonstrate our method on the emission spectrum of HD 189733 b and find good agreement with a traditional retrieval from the Bayesian Atmospheric Radiative Transfer (BART) code (Bhattacharyya coefficients of 0.9843–0.9972, with a mean of 0.9925, between 1D marginalized posteriors). This accuracy comes while still offering significant speed enhancements over traditional RT, albeit not as much as ML methods with lower posterior accuracy. Our method is ∼9× faster per parallel chain than BART when run on an AMD EPYC 7402P central processing unit (CPU). Neural-network computation using an NVIDIA Titan Xp graphics processing unit is 90×–180× faster per chain than BART on that CPU.
BibTeX
@article{himes2022accurate,
  author = {Michael D. Himes and Joseph Harrington and Adam D. Cobb and Atılım Güneş Baydin and Frank Soboczenski and Molly D. O’Beirne and Simone Zorzan and David Wright and Zacchaeus Scheffer and Shawn Domagal‐Goldman and Giada Arney},
  title = {{Accurate Machine-learning Atmospheric Retrieval via a Neural-network Surrogate Model for Radiative Transfer}},
  journal = {The Planetary Science Journal},
  year = {2022},
  doi = {10.3847/psj/abe3fd},
}
Figure