TL;DR
This paper benchmarks different methods for analyzing time series with irregular sampling, a common issue in geosciences. They find that Gaussian kernel-based methods significantly outperform standard linear interpolation and Lomb-Scargle techniques, especially for highly irregular data, providing more accurate estimates of autocorrelation and cross-correlation.
What Problem Does This Paper Address?
Many scientific measurements, particularly in geosciences like paleoclimatology, produce time series that are not evenly spaced. This irregular sampling poses a significant challenge for standard time series analysis techniques, which typically assume regular intervals. Applying these standard methods directly, or using simple linear interpolation to "regularize" the data, can introduce substantial biases, especially in the estimation of correlation functions and the persistence (memory) of the underlying processes. This can lead to misinterpretations of past climate variability and relationships between different climate records. The paper aims to systematically compare various methods designed to handle this irregularity and identify the most robust and accurate approach.
Methods
The authors compare several techniques for estimating autocorrelation functions (ACFs) and cross-correlation functions (CCFs) from irregularly sampled time series:
- Linear Interpolation + FFT: The standard approach of filling in missing data points using linear interpolation and then applying Fast Fourier Transform (FFT)-based methods.
- Lomb-Scargle (LS) Periodogram: A direct transform method designed for unevenly sampled data, originally developed for astronomy.
- Slotting Technique (Rectangular Kernel): A method that bins products of observations based on their time differences.
- Kernel-Based Estimators (Gaussian and Sinc Kernels): These methods use weighted averaging of lagged products, where the weights are determined by a kernel function (e.g., Gaussian or sinc). These kernels smoothly assign importance to observations based on their time difference from the desired lag, avoiding the abrupt cutoff of the slotting method.
The performance of these methods is evaluated using synthetic time series with varying degrees of sampling irregularity (controlled by the skewness of the inter-observation time distribution) and real paleo-climate data. Performance is quantified using the Root Mean Square Error (RMSE), which accounts for both bias and variance.
Datasets
The paper uses two types of datasets for evaluation:
- Synthetic Time Series:
- Sinusoids with random phase (to test basic correlation estimation).
- Autoregressive (AR(1)) processes (to test persistence and coupling).
- Sinusoids with additive red noise (to test performance under different signal-to-noise ratios).
- These synthetic datasets were generated with varying sampling schemes, from regular to highly irregular (using Gamma-distributed inter-observation times with increasing skewness).
- Real Paleo-Data:
- Four speleothem $\delta^{18}$O records from the Asian monsoon domain (Dongge, Heshang, Wanxiang, and Dandak caves) spanning the late Holocene (387-1100 BP). These records exhibit naturally irregular sampling and varying time resolutions.
Key Insights and Ideas
- Irregularity is a Major Bias Source: The degree of sampling irregularity (quantified by skewness) has a dramatic impact on the accuracy of standard methods like linear interpolation. As irregularity increases, bias and RMSE skyrocket.
- Kernel Methods Preserve High Frequencies: Unlike interpolation, kernel-based methods (especially Gaussian) are better at preserving high-frequency components in the signal, which are often lost or biased in traditional approaches.
- Persistence Time is Overestimated by Interpolation: Standard interpolation methods tend to overestimate the persistence (memory) of time series, which can lead to incorrect conclusions about the timescales of climate processes.
- Gaussian Kernel is the Robust All-Rounder: The Gaussian kernel method consistently performs well across various synthetic test cases (sinusoids, AR processes, noise) and is recommended as a reliable and robust estimator.
Key Results
- ACF Estimation: For highly irregular time series, the Gaussian kernel method showed a 40% lower RMSE for the lag-1 ACF compared to linear interpolation.
- CCF Estimation: For highly irregular time series, the Gaussian kernel method showed a 60% lower RMSE for the cross-correlation function (CCF) compared to linear interpolation.
- Lomb-Scargle: Performed comparably to kernel methods for univariate analysis but was poorer for bivariate (cross-correlation) analysis.
- Persistence Time: Interpolation led to a strong overestimation of persistence time (up to a factor of two) for irregular time series, while the Gaussian kernel provided estimates with only a small negative bias compared to a least squares fit on the original record.
- Paleo-Data Application: When applied to Asian monsoon records, cross-correlation results were similar between interpolation and Gaussian kernel methods, attributed to the long timescales of the common variability. However, the Gaussian kernel provided more detailed high-frequency variability in the CCF.
Limitations and Future Work
- Kernel Width Selection: While the authors provide a rationale for scaling kernel widths to the mean sampling rate, they acknowledge that other choices might be sensible depending on the specific time series and statistic being estimated.
- Cross-Spectral Analysis: The paper mentions that kernel-based CCFs show more high-frequency variability, suggesting that cross-spectral analysis could be beneficial, but this is outside the scope of the current study.
- Generalizability: While the Gaussian kernel is recommended, further exploration of other kernel shapes and their specific advantages for different types of geophysical signals could be valuable.
- Software Availability: The authors mention that software for analyzing irregularly sampled time series using these methods is available online, which is a positive step for broader adoption.
Summary updated