Abstract
Observational time series data often exhibit both cyclic temporal trends and autocorrelation and may also depend on covariates. As such, there is a need for flexible regression models that are able to capture these trends and model any residual autocorrelation simultaneously. Modelling the autocorrelation in the residuals leads to more realistic forecasts than an assumption of independence. In this paper we propose a method which combines spline-based semi-parametric regression modelling with the modelling of auto-regressive errors. The method is applied to a simulated data set in order to show its efficacy and to ultrafine particle number concentration in Helsinki, Finland, to show its use in real world problems.
Introduction
Environmental time series, such as air pollution data, are notoriously difficult to model due to cyclic trends (daily and annual), non-linear dependencies with meteorological factors, and inherent autocorrelation. Standard regression models often assume independent errors, which in time-series contexts leads to "over-wiggly" models that fit noise rather than trends, resulting in overconfident and inaccurate forecasts. This paper addresses these challenges by integrating flexible semi-parametric regression (Generalised Additive Models) with formal autoregressive (AR) error structures within a Bayesian framework.
Methodology
The proposed model is a Bayesian Generalised Additive Model (GAM) designed to capture complex dependencies simultaneously.
Model Structure
The hierarchical nature of the model is captured in the Directed Acyclic Graph (DAG) below. It links observed data $y_i$ to mean levels $\mu_i$ influenced by covariates $X$ and spline coefficients $\beta$, while accounting for error terms $\epsilon_i$ governed by an AR process.
Key Components
- Penalised B-splines (P-splines): Used to model non-linear effects with a "smoothness penalty" to prevent overfitting.
- Tensor Product Bases: To capture interactions between variables, such as wind speed and direction, the authors employ 2D splines. This allows the model to learn complex spatial-meteorological relationships.
- Autoregressive (AR) Errors: Residuals are explicitly defined as an AR process, allowing the model to account for persistent deviations from the trend.
- Bayesian Estimation: Markov Chain Monte Carlo (MCMC) methods are used for parameter estimation, providing full probability distributions for forecasts and propagating uncertainty naturally.
Data and Results
The model was validated using simulated data and real-world hourly measurements of Ultrafine Particle Number Concentration (PNC) from Helsinki, Finland.
Key Insights
- Simultaneous Fitting: Fitting trends and autocorrelation simultaneously proved superior to two-step processes, allowing the model to balance smoothness against residual noise effectively.
- Interaction Modeling: Treating wind speed and direction as a joint 2D interaction (tensor product) was significantly more effective than independent treatment, capturing urban transport dynamics.
- Missing Data: The Bayesian framework naturally handled gaps in environmental monitoring data by treating them as unobserved parameters.
Performance and Physical Accuracy
The model achieved the lowest Deviance Information Criterion (DIC) when incorporating joint annual-daily trends and tensor products. It successfully captured known physical phenomena: PNC peaks during rush hours (10:00 AM/PM) and decreases when humidity exceeds 90%. Furthermore, modeling the AR structure significantly reduced residual autocorrelation, leading to more realistic 24- and 48-hour forecasts with credible intervals that grow appropriately over time.
Conclusion
While computationally more intensive than frequentist methods, this Bayesian approach provides a robust framework for environmental forecasting. Future work may involve using Reversible Jump MCMC for automatic knot placement and extending the model to a Generalised Additive Mixed Model (GAMM) for multi-site spatial forecasting.
Summary updated