Nested Sampling for ARIMA Model Selection in Astronomical Time-Series Analysis

Ajinkya Naik, Will HandleyarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces a robust Bayesian framework for astronomical time-series analysis by combining ARIMA models with Nested Sampling. By calculating Bayesian evidence, the method automatically selects the optimal model complexity (order) while providing an inherent "Occam’s penalty" to prevent overfitting, a common pitfall in traditional autoregressive modeling.

What Problem Does This Paper Address?

In the era of high-cadence surveys like LSST, TESS, and Gaia, astronomers are drowning in time-series data. While ARIMA (Autoregressive Integrated Moving Average) models are workhorses in finance and climate science for describing stochastic variability, they are rarely used in astronomy.

The primary hurdle is model selection: an ARIMA model is defined by three integers $(p, d, q)$ representing its autoregressive, integrated, and moving average components. Choosing the "right" numbers is non-trivial. Pick too few, and you miss the physics; pick too many, and you overfit the noise. Traditional heuristics like the Box-Jenkins method or information criteria (AIC/BIC) often struggle with complex likelihood shapes or fail to provide a rigorous measure of model uncertainty.

Methods

The authors bridge this gap by integrating ARIMA with Nested Sampling, a sophisticated numerical algorithm for Bayesian inference.

  • The ARIMA Framework: They model a time series as a combination of its own past values (AR), past forecast errors (MA), and a differencing component (I) to handle non-stationary trends.
  • Nested Sampling: Unlike standard MCMC, Nested Sampling focuses on calculating the Bayesian Evidence ($Z$). This value represents the probability of the data given the model, integrated over all parameters. Crucially, it naturally penalizes models with unnecessary parameters (Occam’s Razor).
  • Vectorization: To make this computationally feasible, the authors implemented a fully vectorized framework using JAX and the blackjax library. This allows the algorithm to run efficiently on modern hardware, performing model selection across entire grids of $(p, q)$ orders simultaneously.
  • Constraints: The sampler enforces mathematical "stationarity" and "invertibility" constraints by checking the roots of characteristic polynomials during the sampling process.

Datasets

The method was validated across four distinct scenarios:

  1. Simulated Data: Artificial AR(2) and ARMA(1,1) series with known parameters to verify that the algorithm can recover the "ground truth."
  2. Sunspot Numbers: The classic yearly sunspot record (1700–2008), a benchmark for time-series analysis.
  3. Kepler Photometry: Lightcurve data from the red giant KIC 12008916, which shows solar-like oscillations.
  4. TESS Photometry: Data from Ross 176, a K-type star hosting a transiting super-Earth, used to test the method's ability to handle gaps and trends in space-based observations.

Key Insights and Ideas

  • Evidence vs. BIC: For the sunspot data, the Bayesian Information Criterion (BIC) favored a simpler ARIMA(3,0,3) model. However, Nested Sampling identified ARIMA(9,0,1) as superior. The higher-order model captured long-memory solar magnetic activity that simpler models missed, leading to significantly better multi-step forecasts.
  • Automated Trend Handling: By treating the "Integrated" ($d$) order as a model selection parameter, the framework correctly identified when a linear trend needed to be removed (differenced) without manual intervention.
  • Parameter Posteriors: Beyond just picking a model, the method provides full posterior distributions for the weights ($\phi, \theta$), giving researchers a clear view of the uncertainties in their physical models.

Key Results

  • Accuracy: In simulated tests, the algorithm correctly identified the true model order with high confidence (e.g., $\log P_{max} = -1.2 \pm 0.5$ for the AR(2) case).
  • Forecasting Performance: For sunspots, the ARIMA(9,0,1) model selected by Nested Sampling achieved a Mean Squared Error (MSE) of 4103.68, compared to 5592.04 for the BIC-favored model—a ~26% improvement in predictive accuracy.
  • Stochastic Modeling: The framework successfully captured the complex, autocorrelated noise in Kepler and TESS lightcurves, effectively "whitening" the residuals (removing temporal structure) which is a prerequisite for high-precision transit detection.

Limitations and Future Work

  • Computational Bottleneck: Sampling valid initial points for very high-order models is difficult because the "stable" region of parameter space becomes extremely small. The authors suggest reparameterizing the coefficients to partial autocorrelations to solve this.
  • Model Extensions: The current work focuses on "classic" ARIMA. The authors point toward extending this to SARIMA (Seasonal), ARFIMA (Fractionally Integrated for long-range dependence), and CARIMA (Continuous-time) models, which are even better suited for irregular astronomical cadences.
  • Physical Interpretation: A major future goal is determining if specific ARIMA structures correlate with specific physical processes (e.g., different types of stellar activity or planetary signals).

Summary updated

Abstract

The upcoming era of large scale, high cadence astronomical surveys demands efficient and robust methods for time series analysis. ARIMA models provide a versatile parametric description of stochastic variability in this context. However, their practical use is limited by the challenge of selecting optimal model orders while avoiding overfitting. We present a novel solution to this problem using a Bayesian framework for time series modelling in astronomy by combining Autoregressive Integrated Moving Average (ARIMA) models with the Nested Sampling algorithm. Our method yields Bayesian evidences for model comparison and also incorporates an intrinsic Occam's penalty for unnecessary model complexity. A vectorized ARIMA Nested Sampling framework is implemented allowing us to perform model selection across grids of Autoregressive (AR) and Moving Average (MA) orders, with efficient inference of selected model parameters. The method is validated on simulated and real astronomical time series, including the yearly sunspots number record, Kepler Lightcurves data of the red giant KIC 12008916, and TESS photometry of the exoplanet host star Ross 176. In all cases, the algorithm correctly identified the true or best-fitting model while simultaneously yielding well constrained posterior distributions for the model parameters. Our results demonstrate that Nested Sampling offers a potentially rigorous alternative to autoregressive model selection in astronomical time series analysis.
BibTeX
@misc{naik2025nested,
  author = {Ajinkya Naik and Will Handley},
  title = {{Nested Sampling for ARIMA Model Selection in Astronomical Time-Series Analysis}},
  year = {2025},
  eprint = {2512.01929},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.01929},
  howpublished = {arXiv},
}
Figure