Conditional Time Series Forecasting with Convolutional Neural Networks

Anastasia Borovykh, Sander M. Bohté, Cornelis W. OosterleearXiv 2017 · DOI · arXiv

Abstract: We present a method for conditional time series forecasting based on an adaptation of the recent deep convolutional WaveNet architecture. The proposed network contains stacks of dilated convolutions that allow it to access a broad range of history when forecasting, a ReLU activation function and conditioning is performed by applying multiple convolutional filters in parallel to separate time series which allows for the fast processing of data and the exploitation of the correlation structure between the multivariate time series. We test and analyze the performance of the convolutional network both unconditionally as well as conditionally for financial time series forecasting using the S&P500, the volatility index, the CBOE interest rate and several exchange rates and extensively compare it to the performance of the well-known autoregressive model and a long-short term memory network. We show that a convolutional network is well-suited for regression-type problems and is able to effectively learn dependencies in and between the series without the need for long historical time series, is a time-efficient and easy to implement alternative to recurrent-type networks and tends to outperform linear and recurrent models.

Introduction

Time series forecasting in financial markets is often considered the "final boss" of predictive modeling. The domain is characterized by non-linear trends, heavy-tailed distributions, and a signal-to-noise ratio that challenges traditional statistical methods. While Recurrent Neural Networks (RNNs), specifically Long Short-Term Memory (LSTM) units, have dominated the field for years due to their ability to handle sequential data, they often suffer from high computational costs and training instability on noisy datasets.

This paper explores an alternative architecture: the Conditional WaveNet (cWN). Originally designed by DeepMind for audio generation, the WaveNet's use of dilated convolutions offers a unique advantage for multivariate financial forecasting by efficiently capturing long-term dependencies without the complexity of recurrent connections.

Methodology

The core innovation lies in adapting the WaveNet architecture for regression-based time series tasks. Unlike standard Convolutional Neural Networks (CNNs) used in computer vision, the cWN employs several specific structural modifications:

Dilated and Causal Convolutions

To capture long-term historical patterns, the model uses dilated convolutions. Instead of processing adjacent data points, the filters "skip" steps at exponentially increasing intervals (e.g., 1, 2, 4, 8). This allows the network's receptive field to grow exponentially with depth, enabling it to look back hundreds of days using only a few layers.

Dilated causal convolutions allow the network to exponentially increase its receptive field with depth, capturing long-term dependencies.

Crucially, these convolutions are causal, ensuring that the prediction at time $t+1$ depends only on information available at time $t$ or earlier, thereby preventing look-ahead bias.

Sparse Connectivity vs. Dense Layers

Compared to standard fully connected (dense) networks, the convolutional approach utilizes sparse connectivity. This reduces the number of parameters, making the model faster to train and less prone to overfitting on the noisy, non-stationary data typical of financial markets.

A comparison between a dense fully connected architecture (left) and the sparse, localized connectivity of a convolutional network (right).

Conditioning and Residual Learning

The model forecasts a target series by conditioning it on related auxiliary series. This is implemented via parallel convolutional filters and parameterized skip connections. These connections allow the model to automatically perform feature selection; if a conditional input (like an interest rate) does not improve the forecast for the target (like an equity index), the network learns to weight that path toward zero. To facilitate training in deep stacks, residual connections are used to prevent vanishing gradients.

Experimental Setup

The authors validated the cWN across three distinct data regimes:

  1. Artificial Data (Lorenz System): A chaotic system used to test the model's ability to recover complex non-linear dynamics in a controlled environment.
  2. Financial Indices: The S&P 500 index, conditioned on the VIX (Volatility Index) and the CBOE 10-year interest rate.
  3. Foreign Exchange (FX): Five major currency pairs (EURUSD, EURJPY, GBPJPY, EURGBP, GBPUSD) tested across three distinct market regimes, including the 2008 financial crisis.

Results and Discussion

The Conditional WaveNet consistently outperformed both traditional Vector Autoregressive (VAR) models and LSTM benchmarks.

  • Accuracy: In the Lorenz system, the cWN achieved an RMSE of 0.00174, significantly lower than the LSTM benchmark of 0.00675. In FX markets, the cWN maintained a Mean Absolute Scaled Error (MASE) well below 1.0, even during high-volatility periods where the VAR model failed (MASE > 1.0).
  • Efficiency: CNNs proved to be more robust for regression-type problems than LSTMs. They were easier to implement, faster to train, and showed a remarkable ability to generalize during market shocks without overfitting to extreme outliers.
  • Feature Selection: The architecture's ability to "ignore" useless conditional inputs through skip connections proved vital for handling the low signal-to-noise ratio of financial data.

Conclusion

Convolutional architectures, specifically those utilizing dilated convolutions, represent a powerful and time-efficient alternative to recurrent networks for financial forecasting. By exploiting the correlation structure between multivariate series and maintaining a broad receptive field, the Conditional WaveNet effectively learns dependencies in short, noisy datasets. Future work may explore the application of this model to high-frequency intraday data or hybrid models that combine autoregressive memory with convolutional non-linear learning.

Summary updated

Abstract

We present a method for conditional time series forecasting based on an adaptation of the recent deep convolutional WaveNet architecture. The proposed network contains stacks of dilated convolutions that allow it to access a broad range of history when forecasting, a ReLU activation function and conditioning is performed by applying multiple convolutional filters in parallel to separate time series which allows for the fast processing of data and the exploitation of the correlation structure between the multivariate time series. We test and analyze the performance of the convolutional network both unconditionally as well as conditionally for financial time series forecasting using the S&P500, the volatility index, the CBOE interest rate and several exchange rates and extensively compare it to the performance of the well-known autoregressive model and a long-short term memory network. We show that a convolutional network is well-suited for regression-type problems and is able to effectively learn dependencies in and between the series without the need for long historical time series, is a time-efficient and easy to implement alternative to recurrent-type networks and tends to outperform linear and recurrent models.
BibTeX
@misc{borovykh2017conditional,
  author = {Anastasia Borovykh and Sander M. Bohté and Cornelis W. Oosterlee},
  title = {{Conditional Time Series Forecasting with Convolutional Neural Networks}},
  year = {2017},
  eprint = {1703.04691},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1703.04691},
}
Figure