Predicting the long-term stability of compact multiplanet systems

Daniel Tamayo, Miles Cranmer, Sam Hadden, Hanno Rein, Peter Battaglia, Alysa Obertas, Philip J. Armitage, Shirley Ho, David N. Spergel, Christian Gilbertson, Naireen Hussain, Ari Silburt, Daniel Jontof‐Hutter, Kristen MenouProceedings of the National Academy of Sciences 2020 · DOI

TL;DR

This paper introduces SPOCK, a machine learning model that can predict the long-term stability of compact multi-planet systems with unprecedented speed and accuracy. By leveraging physically motivated features derived from short simulations, SPOCK dramatically accelerates the process of characterizing exoplanet systems, enabling stronger constraints on their orbital parameters than previously possible.

What Problem Does This Paper Address?

Understanding the long-term stability of planetary systems is a fundamental challenge in astrophysics. While Newton's laws of gravity govern planetary motion, the complex interactions between multiple planets, especially in compact configurations, can lead to chaotic behavior and eventual instability (e.g., collisions or ejections). This is particularly relevant for exoplanet systems, where observational biases often lead us to discover planets in close proximity to their host stars. Accurately predicting stability is crucial for interpreting observational data and constraining planetary properties like eccentricities. Traditional methods, such as direct N-body simulations, are computationally prohibitive for analyzing the vast number of exoplanet candidates, while simpler analytical methods often lack the necessary precision for real-world systems.

Methods

The core of this work is the development of SPOCK, a machine learning classifier based on the XGBoost algorithm. Instead of running computationally expensive long-term simulations (10^9 orbits), SPOCK utilizes a much shorter integration (10^4 orbits) to extract ten "summary statistics." These statistics are derived from physically motivated quantities that capture the system's dynamics, particularly focusing on the influence of mean motion resonances (MMRs). The key innovation lies in the selection and engineering of these features, which are designed to isolate the effects of resonant interactions. SPOCK is trained on a large dataset of simulated three-planet systems, and its performance is then evaluated on various independent datasets, including those with different configurations and multiplicities.

Datasets

  • Training Dataset: Approximately 100,000 three-planet systems, specifically sampled in and near mean motion resonances (MMRs). This dataset was generated to focus on regions of phase space where dynamical behavior changes rapidly.
  • Holdout Test Set (Resonant): 20% of the training dataset, used to test for overfitting.
  • Random Dataset: 25,000 configurations uniformly sampled in period ratios, not necessarily near MMRs. This dataset is more representative of discovered compact exoplanet systems.
  • Higher Multiplicity Systems Dataset: Previously published N-body integrations of 17,500 five-planet systems (equal-mass, co-planar, equally spaced, initially circular orbits) from Obertas et al. (2017).
  • TTV-Constrained Systems: Six systems exhibiting transit timing variations (TTVs), with configurations sampled from MCMC fits to observational data.

Key Insights and Ideas

  • MMRs Drive Instability: The paper strongly suggests that instabilities in compact multi-planet systems are predominantly driven by the overlap of mean motion resonances (MMRs). This insight guided the generation of the training dataset and the selection of features.
  • Feature Engineering is Crucial: The performance of SPOCK hinges on the choice of physically motivated summary statistics, rather than simply feeding raw orbital elements or basic chaos indicators. These features effectively capture the underlying resonant dynamics.
  • Generalization is Key: SPOCK demonstrates remarkable generalization capabilities, performing well on datasets with different configurations (continuous period ratios) and even higher planet multiplicities, despite being trained on specific resonant three-planet systems.
  • Speed vs. Accuracy Trade-off: SPOCK offers a massive speedup (up to 10^5 times) over direct N-body simulations while achieving comparable or superior accuracy to previous methods, making large-scale stability analysis feasible.
  • TTVs Provide Strong Constraints, But SPOCK Offers Complementary Power: While TTVs offer exquisite constraints on orbital parameters, they are often only measurable in systems near strong MMRs. SPOCK provides a powerful alternative for characterizing systems where TTVs are not detectable or informative.

Key Results

  • Speedup: SPOCK achieves speedups of up to 10^5 times compared to direct N-body integrations.
  • Accuracy: On a demanding holdout test set of near-resonant systems, SPOCK correctly classifies 85% of stable systems at a 10% false-positive rate, significantly outperforming previous methods like MEGNO, AMD, and Hill-sphere separation models.
  • Generalization: SPOCK shows strong performance on a uniformly sampled dataset and on higher multiplicity systems, indicating robust generalization.
  • Eccentricity Constraints: For the Kepler-431 system, SPOCK enabled eccentricity constraints below 0.05, which are significantly stronger than those from radial velocity or transit duration measurements and approach TTV-derived constraints.

Limitations and Future Work

  • TTV-Constrained Systems: SPOCK struggles with systems where transit timing variations have already strongly constrained orbital parameters, particularly the resonant dynamics. In such cases, the MCMC fits may narrow down the parameter space to regions where SPOCK's features are less discriminative. The authors suggest that improved models might incorporate additional features to address this.
  • Longer Timescale Instabilities: SPOCK is trained to predict stability over 10^9 orbits and is not designed to capture instabilities driven by slower secular resonances, which can operate on timescales of 10^11-10^12 orbits (e.g., the Solar System's long-term stability).
  • Regression Model: The current SPOCK model performs binary classification (stable/unstable). The authors suggest that developing a regression model to predict precise instability times would be a valuable future direction.
  • Feature Engineering Exploration: The authors are exploring deep learning techniques to generate new features from raw time series data, which could further enhance performance.
  • Public Release: The authors have publicly released the SPOCK code and training dataset to facilitate further research.

Summary updated

Abstract

We combine analytical understanding of resonant dynamics in two-planet systems with machine-learning techniques to train a model capable of robustly classifying stability in compact multiplanet systems over long timescales of [Formula: see text] orbits. Our Stability of Planetary Orbital Configurations Klassifier (SPOCK) predicts stability using physically motivated summary statistics measured in integrations of the first [Formula: see text] orbits, thus achieving speed-ups of up to [Formula: see text] over full simulations. This computationally opens up the stability-constrained characterization of multiplanet systems. Our model, trained on ∼100,000 three-planet systems sampled at discrete resonances, generalizes both to a sample spanning a continuous period-ratio range, as well as to a large five-planet sample with qualitatively different configurations to our training dataset. Our approach significantly outperforms previous methods based on systems' angular momentum deficit, chaos indicators, and parametrized fits to numerical integrations. We use SPOCK to constrain the free eccentricities between the inner and outer pairs of planets in the Kepler-431 system of three approximately Earth-sized planets to both be below 0.05. Our stability analysis provides significantly stronger eccentricity constraints than currently achievable through either radial velocity or transit-duration measurements for small planets and within a factor of a few of systems that exhibit transit-timing variations (TTVs). Given that current exoplanet-detection strategies now rarely allow for strong TTV constraints [S. Hadden, T. Barclay, M. J. Payne, M. J. Holman, <i>Astrophys. J.</i> 158, 146 (2019)], SPOCK enables a powerful complementary method for precisely characterizing compact multiplanet systems. We publicly release SPOCK for community use.
BibTeX
@inproceedings{tamayo2020predicting,
  author = {Daniel Tamayo and Miles Cranmer and Sam Hadden and Hanno Rein and Peter Battaglia and Alysa Obertas and Philip J. Armitage and Shirley Ho and David N. Spergel and Christian Gilbertson and Naireen Hussain and Ari Silburt and Daniel Jontof‐Hutter and Kristen Menou},
  title = {{Predicting the long-term stability of compact multiplanet systems}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2020},
  doi = {10.1073/pnas.2001258117},
}
Figure