Return Prediction for Mean-Variance Portfolio Selection: How Decision-Focused Learning Shapes Forecasting Models

Junhyeong Lee, Haeun Jeon, Hyunglip Bae, Yongjae Lee

TL;DR

Decision-Focused Learning (DFL) improves portfolio optimization by prioritizing final decision quality over raw forecasting accuracy. This paper reveals the underlying mechanism: DFL tilts prediction errors using the inverse covariance matrix, introducing deliberate, strategic biases that overestimate returns for selected assets and underestimate them for excluded ones to maximize portfolio performance.

What Problem Does This Paper Address?

Traditional quantitative finance relies on a two-stage "predict-then-optimize" pipeline. First, a machine learning model is trained to predict asset returns by minimizing a standard loss function like Mean Squared Error (MSE). Then, these predictions are fed into an optimization framework, such as Markowitz's Mean-Variance Optimization (MVO), to allocate portfolio weights.

The flaw in this approach is that MSE treats all prediction errors uniformly across assets. However, MVO is notoriously sensitive to input errors; a tiny miscalculation in the expected return of a highly correlated asset can completely derail the optimal portfolio allocation.

Decision-Focused Learning (DFL) solves this by training the prediction model end-to-end to minimize downstream "regret" (the performance loss of the chosen portfolio compared to the true optimal portfolio). While DFL has consistently demonstrated superior empirical performance, it has largely functioned as a black box. This paper addresses how and why DFL alters forecasting models, explaining why it achieves better financial outcomes despite producing higher raw prediction errors.

Methods

The authors mathematically analyze the gradient of an unconstrained MVO problem under the DFL framework. They discover that while the MSE gradient treats each asset independently, the DFL gradient multiplies the prediction error vector $(\mu^* - \hat{\mu})$ by the inverse covariance matrix $\Sigma^{-1}$.

To study this phenomenon empirically, they train a 4-layer Multi-Layer Perceptron (MLP) to predict expected returns ($\hat{\mu}$) using a combined loss function:

$$\mathcal{L}_{\text{combined}} = \alpha \mathcal{L}_{\text{MVO}} + (1 - \alpha) \mathcal{L}_{\text{MSE}}$$

Here, $\alpha \in [0, 1]$ controls the balance between decision-focus ($\mathcal{L}_{\text{MVO}}$) and prediction accuracy ($\mathcal{L}_{\text{MSE}}$). The end-to-end training pipeline is illustrated below:

Figure 1: DFL training procedure for MVO. The prediction model outputs ˆ𝜇, which determines portfolio weights 𝑤∗( ˆ𝜇) through the optimization layer. The combined loss L𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑= 𝛼L𝑀𝑉𝑂+ (1 −𝛼)L𝑀𝑆𝐸balances prediction accuracy and decision quality. The optimization layer enables computation of 𝑑𝑤∗( ˆ𝜇)

Datasets

The authors evaluate their approach using two long-term historical stock datasets:

  • DOW 30 asset universe
  • S&P 100 asset universe

Both datasets cover a 15-year period from January 1, 2010, to December 31, 2024. Predictions and portfolio optimizations are executed on a daily rolling basis using a 60-day historical look-back window.

Key Insights and Ideas

  • Strategic Bias is a Feature, Not a Flaw: DFL does not attempt to eliminate prediction error. Instead, it intentionally introduces systematic bias to guide the optimizer toward more decisive selections. It overestimates returns for assets it intends to hold ("In group") and underestimates returns for assets it wants to exclude ("Out group").
  • The Inverse Covariance Tilt: The inverse covariance matrix ($\Sigma^{-1}$) acts as a tilting mechanism during gradient descent. It forces the machine learning model to inherently learn inter-asset correlations during the prediction phase, rather than treating assets as independent entities.
  • Extreme Concentration: While MSE-trained models naturally diversify as risk aversion increases, pure DFL models consistently concentrate portfolios to the absolute maximum extent allowed by constraints (utilizing only the minimum number of active assets required).

Key Results

  • Superior Portfolio Performance: Models trained with intermediate DFL weights ($\alpha \in [0.25, 0.75]$) consistently achieved the highest Sharpe ratios, annualized returns, and final wealth across both datasets, significantly outperforming pure MSE ($\alpha = 0$).
  • The Accuracy vs. Decision Trade-off: As the model becomes more decision-focused ($\alpha \to 1$), its MSE loss increases exponentially, yet downstream portfolio regret drops.
  • Polarization of Predictions: Under pure DFL ($\alpha = 1$), top-weighted assets showed persistent positive biases (overestimating returns by 0.5 to 1.5), while bottom-weighted assets showed persistent negative biases (-0.5 to -1.5). This polarization creates a clear separation between selected and excluded assets.

Limitations and Future Work

  • Unconstrained Assumption: The theoretical closed-form analysis relies on an unconstrained MVO framework. Real-world portfolios face budget limits, short-selling constraints, and turnover restrictions, which complicate the gradient landscape and require further study.
  • Asset-Level Granularity: The study focuses primarily on group-level behavior (In vs. Out, Up vs. Down). Future work could explore which specific asset characteristics (e.g., volatility, sector) make them more or less susceptible to DFL's strategic biases.
  • Simpler Alternatives: Because backpropagating through an optimization layer is computationally expensive, the authors suggest exploring whether a simpler hybrid loss function—such as MSE augmented with a static covariance-weighted loss—could mimic DFL's benefits at a fraction of the computational cost.

Summary updated

Abstract

Markowitz laid the foundation of portfolio theory through the mean-variance optimization (MVO) framework. However, the effectiveness of MVO is contingent on the precise estimation of expected returns, variances, and covariances of asset returns, which are typically uncertain. Machine learning models are becoming useful in estimating uncertain parameters, and such models are trained to minimize prediction errors, such as mean squared errors (MSE), which treat prediction errors uniformly across assets. Recent studies have pointed out that this approach would lead to suboptimal decisions and proposed Decision-Focused Learning (DFL) as a solution, integrating prediction and optimization to improve decision-making outcomes. While studies have shown DFL's potential to enhance portfolio performance, the detailed mechanisms of how DFL modifies prediction models for MVO remain unexplored. This study investigates how DFL adjusts stock return prediction models to optimize decisions in MVO. Theoretically, we show that DFL's gradient can be interpreted as tilting the MSE-based prediction errors by the inverse covariance matrix, effectively incorporating inter-asset correlations into the learning process, while MSE treats each asset's error independently. This tilting mechanism leads to systematic prediction biases where DFL overestimates returns for assets included in portfolios while underestimating excluded assets. Our findings reveal why DFL achieves superior portfolio performance despite higher prediction errors. The strategic biases are features, not flaws.
BibTeX
@inproceedings{lee2026return,
  author = {Junhyeong Lee and Haeun Jeon and Hyunglip Bae and Yongjae Lee},
  title = {{Return Prediction for Mean-Variance Portfolio Selection: How Decision-Focused Learning Shapes Forecasting Models}},
  booktitle = {ICAIF-25},
  year = {2026},
  url = {https://openreview.net/forum?id=iXcPvgKkmX},
}

Loading related content...

Figure