Dynamic Portfolio Optimization via Augmented DDPG with Quantum Price Levels-Based Trading Strategy

Runsheng Lin, Zihan Xing, Mingze Ma, Raymond S. T. Lee

Abstract

Dynamic Portfolio Optimization (DPO) involves the real-time rebalancing of asset allocations to maximize returns while mitigating risk. Traditional Deep Reinforcement Learning (DRL) approaches often struggle with high sample complexity and a tendency toward capital over-concentration. This paper introduces an Augmented DDPG framework that integrates Quantum Finance Theory (QFT) with DRL. By utilizing shared encoders, a Gini-based diversification bonus, and Quantum Price Levels (QPLs) for risk control, the proposed model achieves superior risk-adjusted returns compared to standard financial and DRL baselines.

Introduction

In the volatile landscape of financial markets, DPO remains a significant challenge. While DRL agents can learn complex market dynamics directly from raw data, they are often hindered by the noise inherent in financial time series and the lack of explicit risk management mechanisms. This research addresses these hurdles by "augmenting" the Deep Deterministic Policy Gradient (DDPG) algorithm with physical principles from quantum finance and structural optimizations for faster convergence.

Methodology

The Augmented DDPG framework consists of four primary innovations designed to enhance both learning efficiency and trading performance:

1. Integrated Actor-Critic Structure

Standard DDPG implementations utilize separate networks for the Actor (policy) and the Critic (value estimation). The proposed model employs a shared CNN-based state encoder. By sharing parameters, the model learns a more robust representation of market states more quickly, effectively reducing sample complexity and training time.

2. Quantum Price Levels (QPLs)

Drawing from Quantum Finance Theory, the model treats financial assets as particles exhibiting wave-particle duality. By solving the Quantum Finance Schrodinger Equation (QFSE) using an anharmonic oscillator model, the system identifies "energy levels" that serve as dynamic support and resistance lines. These QPLs provide a non-lagging physical basis for identifying market reversals.

K-line analysis of CNY/USD showing price interactions with Quantum Price Levels (QPLs) across different dates in 2021.

3. PG Risk Control Agent

A secondary Policy Gradient (PG) agent, utilizing a Gated Recurrent Unit (GRU) encoder, monitors price action relative to these QPLs. As shown in the figure above, when prices penetrate specific quantum levels (scenarios a-d), the agent generates bullish or bearish signals to trigger early trade executions, allowing the system to capitalize on peaks and troughs more effectively than traditional technical indicators.

4. Gini Bonus for Diversification

To solve the "all eggs in one basket" problem common in DRL, a Gini Bonus is added to the reward function. This mathematical incentive encourages the agent to maintain a balanced portfolio, effectively embedding classical diversification theory into the reinforcement learning process.

Experimental Results

The model was evaluated using historical data for five major Forex pairs (CNY/USD, EUR/GBP, EUR/JPY, EUR/USD, and GBP/JPY) from 2014 to 2021. Back-testing was conducted on the final 20% of the dataset (July 2020 – January 2022).

Metric Augmented DDPG Original DDPG Online Newton Step (ONS) Best Asset
Annual Rate of Return (ARR) 1.2104 0.9999 1.0387 1.1011
Sharpe Ratio (ASR) 1.0811 0.0196 0.3129 0.3398
Max Drawdown (MDD) 1.7695% 9.2718% 3.1938% 5.6688%

The Augmented DDPG model significantly outperformed all baselines. Most notably, it maintained a remarkably low Max Drawdown of 1.77%, compared to over 9% for the standard DDPG, demonstrating the efficacy of the QPL-based risk control mechanism.

Conclusion and Future Work

The integration of Quantum Finance Theory with DRL provides a powerful framework for dynamic portfolio management. While the current model excels in price-action analysis, future work will explore the inclusion of fundamental data and Natural Language Processing (NLP) to capture market sentiment. Additionally, refining execution strategies to account for real-world slippage and liquidity constraints will be critical for high-frequency deployment.

Summary updated

Abstract

With the development of deep learning, Dynamic Portfolio Optimization (DPO) problem has received a lot of attention in recent years, not only in the field of finance but also in the field of deep learning. Some advanced research in recent years has proposed the application of Deep Reinforcement Learning (DRL) to the DPO problem, which demonstrated to be more advantageous than supervised learning in solving the DPO problem. However, there are still certain unsolved issues: 1) DRL algorithms usually have the problems of slow learning speed and high sample complexity, which is especially problematic when dealing with complex financial data. 2) researchers use DRL simply for the purpose of obtaining high returns, but pay little attention to the problem of risk control and trading strategy, which will affect the stability of model returns. In order to address these issues, in this study we revamped the intrinsic structure of the model based on the Deep Deterministic Policy Gradient (DDPG) and proposed the Augmented DDPG model. Besides, we also proposed an innovative risk control strategy based on Quantum Price Levels (QPLs) derived from Quantum Finance Theory (QFT). Our experimental results revealed that our model has better profitability as well as risk control ability with less sample complexity in the DPO problem compared to the baseline models.
BibTeX
@misc{lin2023dynamic,
  author = {Runsheng Lin and Zihan Xing and Mingze Ma and Raymond S. T. Lee},
  title = {{Dynamic Portfolio Optimization via Augmented DDPG with Quantum Price Levels-Based Trading Strategy}},
  year = {2023},
  eprint = {2501.08528},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.08528},
}
Figure