OFTER: An Online Pipeline for Time Series Forecasting

Nikolas Michael, Mihai Cucuringu, Sam HowisonarXiv 2023 · OpenReview

Abstract

We introduce OFTER, a time series forecasting pipeline tailored for mid-sized multivariate time series. OFTER utilizes the non-parametric models of k-nearest neighbors and Generalized Regression Neural Networks, integrated with a dimensionality reduction component. To circumvent the curse of dimensionality, we employ a weighted norm based on a modified version of the maximal correlation coefficient. The pipeline we introduce is specifically designed for online tasks, has an interpretable output, and is able to outperform several state-of-the art baselines. The computational efficacy of the algorithm, its online nature, and its ability to operate in low signal-to-noise regimes, render OFTER an ideal approach for financial multivariate time series problems, such as daily equity forecasting. Our work demonstrates that while deep learning models hold significant promise for time series forecasting, traditional methods carefully integrating mainstream tools remain very competitive alternatives with the added benefits of scalability and interpretability.

Introduction

While deep learning models like LSTMs and Transformers have revolutionized time series forecasting, they often struggle with "mid-sized" datasets where the number of observations is limited. In these scenarios, high-parameter models are prone to overfitting, especially in "low signal-to-noise" regimes (e.g., predicting stock returns).

On the other hand, simple non-parametric methods like k-Nearest Neighbors (kNN) and Generalized Regression Neural Networks (GRNN) are fast and interpretable but suffer from the curse of dimensionality: as the number of input features grows, the distance between points becomes less meaningful, causing performance to collapse. OFTER (Online Forecasting via Temporal Embedding Regression) is a novel, computationally efficient pipeline designed to bridge this gap by creating a system that handles high-dimensional data efficiently in an online (real-time) setting.

Methods

The OFTER pipeline consists of four main stages, designed to process data from raw features to a final ensemble forecast in an online fashion.

The OFTER pipeline architecture, illustrating the flow from raw features through dimensionality reduction and feature selection to the final model ensemble.
  1. Preprocessing & Residualization: The target variable is first modeled using a univariate ARIMA process to capture linear temporal dependencies. The "innovation" (the part ARIMA can't explain) is passed to the machine learning component.
  2. Online Dimensionality Reduction: To handle high-dimensional feature sets, the paper uses Principal Component Analysis (PCA). Crucially, they implement a rank-one update method (solving the "secular equation"). This allows the model to update its internal representation of the data as new points arrive without re-calculating the entire PCA from scratch, making it truly "online."
  3. Feature Weighting via OSMC: To combat the curse of dimensionality, the authors introduce One-Sided Maximal Correlation (OSMC). Using Bernstein polynomials, they transform features to maximize their correlation with the target. These correlations are then used as weights in a weighted Euclidean distance metric, ensuring the kNN/GRNN models focus on the most predictive features.
  4. Dynamic Model Selection: The pipeline maintains an ensemble of kNN, GRNN, and OLS models with various hyperparameters. It tracks the historical loss of each and automatically selects the best-performing configuration for the current time step.

Datasets

The authors evaluated OFTER across three distinct types of data:

  • Synthetic Data: Three 5-dimensional toy models (Linear, Non-linear, and Vector Autoregressive) designed to simulate complex signals like EEG data.
  • Financial Equity Data: A universe of 210 instruments, including 10 ETFs (like SPY and XLF) and 200 financial sector equities. The task was to forecast daily returns and trading volumes.
  • Beijing Air Quality: A public dataset from the UCI repository containing pollution concentrations (PM2.5, PM10, etc.) and weather variables across 12 monitoring stations.

Key Results and Insights

OFTER demonstrates that for mid-sized multivariate tasks, "traditional" non-parametric methods—when carefully integrated with modern dimensionality reduction—remain very competitive against deep learning.

Performance and Speed

In financial return forecasting, OFTER-DR (the dimensionality reduction variant) achieved an annualized Sharpe Ratio of 1.616, significantly outperforming the best deep learning baseline (GRU at 0.424) and Random Forests (0.371). Furthermore, OFTER is orders of magnitude faster than deep learning. On the financial dataset, OFTER took roughly 17-22 seconds to run, while LSTM and GRU required 442-549 seconds.

Interpretability and Robustness

Unlike "black-box" neural networks, OFTER provides explicit feature importance scores. By analyzing the sensitivity of the weighted distance metric, users can see exactly which input variables are driving a specific forecast. Additionally, the model showed consistent performance across various hyperparameter settings, suggesting it is less finicky to tune than deep neural networks.

Limitations and Future Work

The authors acknowledge that OFTER currently struggles to discover non-linear interaction effects between covariates (e.g., when the interaction of Feature A and Feature B is predictive, but neither is predictive alone). Future work suggests integrating a "pre-screening" component to test for and include interaction terms in the feature space and extending the pipeline to probabilistic forecasting to provide uncertainty estimates alongside point forecasts.

Summary updated

Abstract

We introduce OFTER, a time series forecasting pipeline tailored for mid-sized multivariate time series. OFTER utilizes the non-parametric models of k-nearest neighbors and Generalized Regression Neural Networks, integrated with a dimensionality reduction component. To circumvent the curse of dimensionality, we employ a weighted norm based on a modified version of the maximal correlation coefficient. The pipeline we introduce is specifically designed for online tasks, has an interpretable output, and is able to outperform several state-of-the art baselines. The computational efficacy of the algorithm, its online nature, and its ability to operate in low signal-to-noise regimes, render OFTER an ideal approach for financial multivariate time series problems, such as daily equity forecasting. Our work demonstrates that while deep learning models hold significant promise for time series forecasting, traditional methods carefully integrating mainstream tools remain very competitive alternatives with the added benefits of scalability and interpretability.
BibTeX
@misc{michael2023ofter,
  author = {Nikolas Michael and Mihai Cucuringu and Sam Howison},
  title = {{OFTER: An Online Pipeline for Time Series Forecasting}},
  year = {2023},
  eprint = {2304.03877},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2304.03877},
}
Figure