Extending Deep Learning Models for Limit Order Books to Quantile Regression

Zihao Zhang, Stefan Zohren, Stephen RobertsarXiv 2019 · DOI · arXiv

Abstract

We showcase how Quantile Regression (QR) can be applied to forecast financial returns using Limit Order Books (LOBs), the canonical data source of high-frequency financial time-series. We develop a deep learning architecture that simultaneously models the return quantiles for both buy and sell positions. We test our model over millions of LOB updates across multiple different instruments on the London Stock Exchange. Our results suggest that the proposed network not only delivers excellent performance but also provides improved prediction robustness by combining quantile estimates.

Introduction

Predicting financial returns in high-frequency environments is a formidable challenge. Market data is inherently non-stationary, exhibits high kurtosis ("fat tails"), and is subject to significant noise. Traditional linear models, such as Vector Autoregression (VAR), often struggle with these characteristics due to rigid parametric assumptions. While modern deep learning approaches have improved point-estimation for future prices, a single "average" prediction is insufficient for sophisticated risk management. Traders require an understanding of the full distribution of potential outcomes to manage risk exposure effectively.

Furthermore, most existing research focuses on "mid-price" returns. However, the mid-price is a theoretical construct that ignores the bid-ask spread—the actual cost of executing a trade. This paper introduces DeepLOB-QR, an architecture designed to predict the actual tradable returns of long and short positions while providing a probabilistic measure of uncertainty through quantile regression.

Methodology

The core of the proposed approach is the integration of Quantile Regression (QR) into a deep learning framework. Unlike standard regression that minimizes Mean Squared Error to estimate the mean, QR minimizes the "pinball loss" (Quantile Loss) to estimate specific percentiles of the target distribution.

DeepLOB-QR Architecture

The model utilizes a hybrid architecture combining Convolutional Neural Networks (CNNs) for spatial feature extraction from the LOB and Long Short-Term Memory (LSTM) units for capturing temporal dependencies.

  • Shared Backbone: A shared convolutional block processes raw LOB data to extract universal features.
  • Multi-Output Heads: The network branches into multiple LSTM layers to simultaneously estimate six outputs: the 25th, 50th, and 75th quantiles for both "long" and "short" positions.
  • Return Definitions: The model predicts actual tradable returns rather than mid-price changes. Figure 4 illustrates how these returns are defined relative to the bid-ask spread and the cost of crossing the book.
Geometric definition of long and short returns relative to the bid-ask spread.

Robustness and Constraints

To ensure logical consistency, the model employs a rearrangement move to prevent "quantile crossing" (e.g., ensuring the 75th percentile is always greater than the 50th). Additionally, the authors introduce DeepLOB-QR(C), which uses a constrained optimization technique to combine individual quantile estimates into a single, highly robust point forecast.

Data and Experimental Setup

The study leverages a massive dataset from the London Stock Exchange (LSE) covering the full year of 2017. The data includes five highly liquid stocks: Lloyds, Barclays, Tesco, BT, and Vodafone, totaling over 134 million observations. For each timestamp, the model inputs 10 levels of price and volume for both the bid and ask sides, resulting in a 40-dimensional feature vector.

Key Insights: The Reality of Market Friction

A central finding of this research is that mid-price returns are often misleading. Because they ignore the bid-ask spread, mid-price returns can appear significantly higher than the returns a trader would actually realize.

Comparison of mid-price returns versus actual long and short returns, highlighting the impact of the bid-ask spread.

As shown in Figure 1, the mid-price return ($r_{mid,t}$) can be substantially larger than the long return ($r_{long,t}$). By modeling long and short positions separately, DeepLOB-QR captures the inherent asymmetry of buying and selling in a live market.

Results

DeepLOB-QR was benchmarked against standard models including Autoregressive (AR) models, Support Vector Regression (SVR), and standard Multi-Layer Perceptrons (MLP).

  • Accuracy: DeepLOB-QR(C) consistently achieved the lowest Mean Absolute Error (MAE) and Mean Squared Error (MSE).
  • Explanatory Power: The model yielded the highest $R^2$ scores for both long and short positions across various prediction horizons.
  • Distributional Information: Interestingly, the weighted combination of multiple quantiles (DeepLOB-QR(C)) outperformed a model trained solely to predict the median. This indicates that the "shape" of the predicted distribution contains valuable information for improving point estimates.

Conclusion and Future Directions

DeepLOB-QR demonstrates that modeling the full distribution of returns provides superior predictive power and risk awareness compared to traditional point-estimation methods. While performance naturally degrades as the prediction horizon increases, the model remains robust.

Future research could explore denser quantile sets for more granular risk assessment. Furthermore, the uncertainty metrics provided by DeepLOB-QR could be integrated into Reinforcement Learning (RL) agents to optimize the balance between exploration and exploitation in automated trading strategies.

Summary updated

Abstract

We showcase how Quantile Regression (QR) can be applied to forecast financial returns using Limit Order Books (LOBs), the canonical data source of high-frequency financial time-series. We develop a deep learning architecture that simultaneously models the return quantiles for both buy and sell positions. We test our model over millions of LOB updates across multiple different instruments on the London Stock Exchange. Our results suggest that the proposed network not only delivers excellent performance but also provides improved prediction robustness by combining quantile estimates.
BibTeX
@misc{zhang2019extending,
  author = {Zihao Zhang and Stefan Zohren and Stephen Roberts},
  title = {{Extending Deep Learning Models for Limit Order Books to Quantile Regression}},
  year = {2019},
  eprint = {1906.04404},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.04404},
}
Figure