Abstract
Sequential data with serial correlation and an unknown, unstructured, and dynamic background is ubiquitous in neuroscience, psychology, and econometrics. Inferring serial correlation for such data is a fundamental challenge in statistics. We propose a total variation constrained least square estimator coupled with hypothesis tests to infer the serial correlation in the presence of unknown and unstructured dynamic background. The total variation constraint on the dynamic background encourages a piece-wise constant structure, which can approximate a wide range of dynamic backgrounds. The tuning parameter is selected via the Ljung-Box test to control the bias-variance trade-off. We establish a non-asymptotic upper bound for the estimation error through variational inequalities. We also derive a lower error bound via Fano's method and show the proposed method is near-optimal. Numerical simulation and a real study in psychology demonstrate the excellent performance of our proposed method compared with the state-of-the-art.
Introduction
In fields like neuroscience, psychology, and econometrics, researchers often grapple with "messy" time-series data. Imagine measuring a person's reaction time over an hour-long experiment. A primary goal is often to determine if their current response is correlated with their previous one—a property known as serial correlation. However, such data is frequently "contaminated" by a dynamic background: the subject might experience fatigue, learning effects, or fluctuations in attention. This non-stationary drift is a significant hurdle for standard statistical models, as it often masquerades as actual correlation, leading to biased and unreliable results.
The Problem of Non-Stationarity
The core issue in analyzing sequential human or economic data is non-stationarity. Standard autoregressive (AR) models assume the underlying process is stable over time. However, if a subject’s reaction time gradually slows down due to fatigue, a standard model might interpret that slow-moving trend as high serial correlation. Conversely, attempting to "clean" the data by over-fitting the background can accidentally erase the very correlation researchers are trying to measure. Until now, there has been a lack of efficient, principled methods to simultaneously estimate both correlation coefficients and highly unstructured background drift.
The TV-Constrained Estimator
To address this, we propose a Total Variation (TV) Constrained Least Square Estimator. This approach models the data as a combination of a dynamic background, an autoregressive component (the serial correlation), and random noise.
By imposing a TV constraint on the background, the model is encouraged to find a piecewise constant structure. This structural assumption is flexible enough to approximate almost any unstructured drift without requiring a specific functional form (like a linear or polynomial trend). A critical component of this method is selecting the right "budget" (denoted as $\delta$) for the background's variation. We utilize the Ljung-Box test—a standard statistical test for remaining correlation in residuals—to identify the optimal $\delta$ that captures the correlation without overfitting the noise.
Theoretical Foundations and Optimality
Beyond empirical performance, we establish the theoretical rigor of this approach. We derived a non-asymptotic upper bound for the estimation error and utilized Fano’s method to establish a lower error bound. These results demonstrate that the proposed method is near-optimal, meaning its error rate stays close to the theoretical minimum even in finite time horizons.
Empirical Performance
Synthetic Validation
The method was validated using simulated AR(1) and AR(2) sequences with various complex, oscillating backgrounds. In head-to-head comparisons, the TV-constrained method consistently outperformed polynomial approximations and other state-of-the-art "fused" estimators, particularly when the background drift was highly irregular.
Real-World Psychology Application
We applied the method to a Reaction Time (RT) dataset involving 28 subjects performing 1,000 trials of a perceptual judgment task. Standard AR(1) models consistently yielded significantly inflated correlation estimates because they confused slow-moving fatigue and attention drifts for actual trial-to-trial correlation. Our method revealed that while most subjects showed serial dependence, the intensity and shape of their background drift varied wildly—some were nearly stationary, while others exhibited complex waves of attention.
Conclusion
This work provides a vital new tool for researchers who need to see through the "fog" of non-stationary drift. By formalizing background drift as a nuisance variable that can be modeled via piecewise constants, we offer a robust way to find the true patterns of dependence in sequential data. While computationally more intensive than simple regression, the gain in accuracy and the correction of inherent biases make it a necessary advancement for high-stakes data analysis in psychology and neuroscience.
Summary updated