Abstract
We describe a family of conservative statistical tests for independence of two autocorrelated time series. The series may take values in any sets, and one of them must be stationary. A user-specified function quantifying the association of a segment of the two series is compared to an ensemble obtained by time-shifting the stationary series -N to N steps. If the series are independent, the unshifted value is in the top m shifted values with probability at most m/(N+1). For large N, the probability approaches m/(2N+1). A conservative test rejects independence at significance α if the unshifted value is in the top α(N+1), and has half the power of an approximate test valid in the large N limit. We illustrate this framework with a test for correlation of autocorrelated categorical time series.
Introduction
In statistical analysis, the assumption that observations are independent and identically distributed (i.i.d.) is fundamental but frequently violated by time series data. In such data, autocorrelation—where a value at time $t$ depends on previous states—can lead standard tools like Pearson’s correlation or Fisher’s exact test to "hallucinate" relationships. This phenomenon, known as spurious correlation, occurs because these tests do not account for the internal structure of the series. While "pre-whitening" techniques exist to remove these patterns, they are often difficult to apply to non-linear or categorical data. This paper introduces the Shift Test, a framework that provides mathematically guaranteed conservative bounds on false positives for any data type.
Methodology: The Shift Test
The Shift Test operates on a simple yet powerful principle: if two series are genuinely related, their association should be strongest when they are correctly aligned in time. If they are independent, the association at zero shift should be indistinguishable from the association at random time offsets.
To perform the test:
- Define an Association Function ($V$): This can be any metric, such as correlation, log-odds ratio, or classifier accuracy.
- Create an Ensemble: Compare a central segment of series $X$ to series $Y$ at its original position (shift 0) and at $2N$ different time-shifted positions (from $-N$ to $N$).
- Rank the Results: Determine the rank $m$ of the unshifted association value within the ensemble.
- Statistical Inference: Reject the null hypothesis of independence if $m$ is below a threshold determined by the desired significance level $\alpha$.
As long as one of the series is stationary, this test is guaranteed to be conservative, meaning the false positive rate will not exceed $\alpha$.
Experimental Evaluation
The framework was evaluated using synthetic categorical time series generated via Markov chains. Two scenarios were tested: independent chains (highly autocorrelated but unrelated) and correlated chains (sharing common states).
Key Results
- Failure of Standard Tests: When applied to independent but autocorrelated data, Fisher’s exact test produced a massive peak of "significant" p-values (Panel B, left), indicating a high rate of false positives.
- Robustness of the Shift Test: The Shift Test correctly maintained the null hypothesis for independent series. For correlated series, the association function $V$ (cross-table log odds) displayed a sharp, distinct peak at zero shift (Panel C, right), allowing the test to detect the relationship in 86.9% of simulations with $N=19$.
- Data Agnosticism: The test successfully handled categorical data without requiring Gaussian assumptions or complex modeling of the underlying process.
Discussion and Limitations
The Shift Test's primary strength is its flexibility; it works for generic time series as long as an association function can be defined. However, it requires at least one series to be stationary. If both series exhibit strong non-stationary trends, spurious results may still occur. Additionally, the test's statistical power depends on the choice of $V$ and the shift range $N$. Users must choose $N$ to be large enough to capture the autocorrelation timescale but small enough to maintain sufficient data for comparison. Finally, the association function must be chosen independently of the test data to preserve statistical validity.
Summary updated