QADQN: Quantum Attention Deep Q-Network for Financial Market Prediction

Siddhant Dutta, Nouhaila Innan, Alberto Marchisio, Sadok Ben Yahia, Muhammad Shafique

Introduction

Predicting financial market trends is a formidable challenge characterized by high volatility, non-stationary data, and the intricate interplay of global economic variables. While classical Deep Learning architectures, such as LSTMs and Transformers, have significantly advanced predictive modeling, they often encounter limitations in computational efficiency and a susceptibility to overfitting noisy financial signals. Furthermore, Reinforcement Learning (RL) agents in trading frequently struggle with the "cold start" problem—the period of initial exploration where random actions can lead to substantial capital loss.

This paper introduces the Quantum Attention Deep Q-Network (QADQN), a hybrid framework that integrates quantum-enhanced attention mechanisms with reinforcement learning. By leveraging the high-dimensional processing capabilities of Variational Quantum Circuits (VQCs), the model aims to improve feature extraction and decision-making in complex market environments.

Methodology

The QADQN architecture utilizes a multi-stage hybrid pipeline designed to capture temporal dependencies while exploiting quantum advantages for attention and policy output.

1. Hybrid Architecture

  • Classical Pre-Net (LSTM): The model first processes historical Open-High-Low-Close (OHLC) price data through a Long Short-Term Memory (LSTM) network. This stage is responsible for capturing the sequential temporal dependencies inherent in financial time series.
  • Quantum Multi-Head Self-Attention (QMSA): This represents the core innovation of the framework. In place of classical matrix multiplications, the model encodes data into quantum states using parameterized rotation gates. It calculates Key, Query, and Value matrices through quantum measurements, allowing the model to navigate high-dimensional feature relationships more efficiently than classical attention layers.
  • Quantum Post-Net Layer: A VQC serves as the final decision-making component, mapping the processed features to Q-values that represent the expected utility of three possible actions: buy, sell, or hold.

2. Training and Optimization

To address the exploration-exploitation dilemma, the authors implement Imitative Learning. The agent's experience replay buffer is pre-populated with actions derived from the "Dual Thrust" strategy, a classical trend-following system. This jumpstarts the agent's expertise, reducing the risks associated with early-stage random exploration.

Optimization is handled by the Lion (EvoLved Sign Momentum) optimizer, chosen for its memory efficiency, and the Huber loss function, which provides robustness against the outliers common in volatile market data.

Experimental Setup

The researchers evaluated QADQN using historical data from the S&P 500 index (via Yahoo Finance).

  • Training Period: January 15, 2016, to January 15, 2020.
  • Testing Periods: The model was tested on both an overlapping period (2019–2024) and a non-overlapping period (2020–2024) to ensure the results were not a byproduct of data leakage.
  • Realistic Constraints: To simulate real-world conditions, the study incorporated a 0.2% transaction cost per trade, a factor often omitted in theoretical trading models.

Key Results and Discussion

QADQN demonstrated significant performance gains over traditional benchmarks, including the standard Buy & Hold strategy and the classical Deep Deterministic Policy Gradient (DDPG) model.

Financial Performance

In the non-overlapping test period (2020–2024), QADQN achieved a 71.26% total return, substantially outperforming Buy & Hold (44.22%) and DDPG (15.94%). The model's ability to identify market lows for entry and peaks for exit was a primary driver of this alpha generation.

Risk Management

Beyond absolute returns, the model excelled in risk-adjusted metrics:

  • Sortino Ratio: The model achieved a Sortino ratio of 1.28, indicating superior returns relative to downside volatility.
  • Maximum Drawdown: QADQN limited its maximum drawdown to 19.24%, successfully navigating the extreme market volatility of early 2020 more effectively than its classical counterparts.

Conclusion and Future Directions

The QADQN framework suggests that quantum-enhanced attention mechanisms can provide a measurable edge in financial time-series prediction. By combining the temporal processing of LSTMs with the high-dimensional expressivity of quantum circuits, the model achieves superior risk-adjusted returns while mitigating the "cold start" risks of RL.

However, several challenges remain for live deployment. The current 4-qubit configuration must be scaled to handle multi-asset portfolios, and the impact of quantum noise on Noisy Intermediate-Scale Quantum (NISQ) devices requires further study. Future research will focus on integrating quantum error correction and expanding the model's application to more diverse asset classes such as forex and cryptocurrencies.

Summary updated

BibTeX
@inproceedings{dutta2024qadqn,
  author = {Siddhant Dutta and Nouhaila Innan and Alberto Marchisio and Sadok Ben Yahia and Muhammad Shafique},
  title = {{QADQN: Quantum Attention Deep Q-Network for Financial Market Prediction}},
  booktitle = {QCE},
  year = {2024},
  doi = {10.1109/qce60285.2024.10303},
}
Figure