Imagine you are operating a day-trading algorithm. The price of an asset is currently \$100, and tomorrow it will rise to \$105.
You evaluate two predictive models:
- Model A predicts the price will be \$99.
- Model B predicts the price will be \$111.
From a traditional statistical standpoint, both models are equally flawed—each is exactly \$6 away from the ground truth. However, from an operational standpoint, their utility could not be more different. Model B correctly identifies that the market is moving up, prompting a "Buy" decision that yields a profit. Model A suffers from a sign error, predicting a downward move that results in a costly short position.
This scenario illustrates the core challenge of metric-utility misalignment in time series forecasting. Standard statistical error metrics, such as Mean Squared Error (MSE) or Mean Absolute Percentage Error (MAPE), measure the geometric distance between a prediction and a target. Yet, in real-world applications like quantitative finance, energy trading, or inventory management, the value of a forecast depends on its actionability—specifically, its ability to correctly capture directional trends and high-impact anomalies. As Bitvai et al. observe, a model can achieve exceptional numerical proximity to a target while remaining completely useless, or even financially damaging, if it consistently misjudges the side of a price movement.
The Mimicking Trap: Why Models Cheat to Minimize Error
To understand why standard models fail downstream, we must look at how neural networks optimize for distance-based losses like MSE. When a model is trained on highly volatile, noisy, or near-random-walk data (such as stock prices), the mathematically "safest" way to minimize squared error is to output a value very close to the most recent observation.
Kosma et al. formally define this behavior as mimicking. Instead of learning the underlying dynamics of the system, the model simply "copies the past" and lags one step behind the true trend.
Price
^ /<- True Trend
| /
| / .-- Mimicking Forecast (Low MSE, but always lags behind)
| / /
+------------------------> Time
This strategy yields a deceptively low average error on paper, but it provides zero directional insight. Zhang et al. demonstrate that point-wise error metrics are fundamentally incapable of distinguishing between a model that genuinely uncovers market trends and one that merely predicts "no change."
Furthermore, standard metrics treat all errors symmetrically. In practice, business utility is highly asymmetric. In customer retention scenarios, Höppner et al. note that treating all classification errors as equal fails to account for the varying financial impacts of different retention outcomes. In financial markets, Cuzzolin et al. argue that aggregate metrics like MAPE are decoupled from profitability because they fail to account for compounding returns and extreme outliers. A model might be highly precise during quiet market periods but completely miss the critical, high-impact spikes that drive the majority of long-term returns or losses—a limitation also highlighted by Eugene W Park in his analysis of market dynamics.
The Directional Accuracy Trade-Off
An elegant insight emerging from recent research is that optimizing for precise point estimation and optimizing for trend direction are often fundamentally at odds.
Zhiyang Chen uncovered a distinct trade-off when evaluating deep learning architectures: while traditional recurrent networks might minimize raw numerical variance, Transformer-based models often exhibit superior capabilities in identifying the direction of market trends, making them far more effective for strategy-based trading despite higher absolute errors.
This divergence means that a model can be an excellent regressor but a terrible decision-maker. As single-author studies by Wanqing Li (2021) and Han Li (2023) independently show, high precision in predicting specific price points does not correlate with accuracy in predicting the direction of market movement. Because point-accuracy and trend-accuracy require entirely different optimization landscapes, models trained purely on distance metrics frequently fail to generate actionable insights.
Mechanisms for Aligning Models with Utility
To bridge the gap between statistical precision and operational utility, researchers have developed several distinct methodologies that force models to prioritize directional correctness and decision-making value.
1. Reformulating Regression as Trend Classification
Instead of forcing a network to predict exact future magnitudes—which introduces high variance and invites mimicking—the problem can be reframed as predicting discrete directional states.
Dong et al. demonstrate that treating return forecasting as a multi-state classification problem (e.g., classifying movements into upward, downward, or flat bins) significantly improves risk-adjusted performance. This formulation explicitly minimizes sign errors, protecting portfolios from catastrophic wrong-way bets. Taking this a step further, Sawhney et al. bypass price forecasting altogether by utilizing reinforcement learning agents that optimize directly for portfolio returns and risk management via discrete "Buy, Sell, or Hold" actions.
2. Direction-Penalizing Loss Functions
If a regression framework is necessary, the loss function can be modified to penalize directional mistakes more severely than magnitude mistakes.
The authors of Ye et al. introduce specialized loss functions that penalize prediction errors exponentially when the predicted direction contradicts the actual market movement:
$$\text{Loss} = \begin{cases} |\hat{y}_t - y_t| & \text{if } \text{sign}(\hat{y}_t - y_{t-1}) = \text{sign}(y_t - y_{t-1}) \\ e^{|\hat{y}_t - y_t|} & \text{if } \text{sign}(\hat{y}_t - y_{t-1}) \neq \text{sign}(y_t - y_{t-1}) \end{cases}$$
By forcing the network to prioritize the correct sign over raw numerical variance, the model aligns more closely with profitable execution. Similarly, Ji et al. integrate a dedicated trend-prediction term into a hybrid loss function for Transformer decoding, preventing the model from becoming "trend-blind" during multi-step forecasting.
3. Parity Calibration
Even when models output directional probabilities, their confidence intervals are often poorly calibrated relative to the trend. Chung et al. introduce parity calibration, a framework that explicitly treats directional trends as a binary classification problem. By ensuring that a model's stated confidence in a directional move matches its empirical frequency (e.g., an 80% predicted chance of an increase actually happens 80% of the time), they prevent the common pitfall where a model provides tight confidence intervals for an absolute value while being consistently wrong about its direction.
4. Correlation-Aware and Pattern-Level Evaluation
Finally, evaluating models requires metrics that mirror downstream utility. Zhang et al. introduce correlation-aware metrics (msIC and msIR) that focus on the rank correlation between predicted and actual movements, serving as a much stronger predictor of trading profitability than MSE. On a broader scale, the next-generation time series benchmark introduced by Qiao et al. shifts away from aggregate distance metrics toward pattern-level evaluation frameworks, explicitly testing a model's ability to capture critical spikes and directional changes that dictate real-world operational success.
Key Foundational Work
A Second Correlation Method for Multivariate Exchange Rates Forecasting (2014) — contributed 5 concept elements to this direction.
Day trading profit maximization with multi-task learning and technical analysis (2015) — contributed 1 concept elements to this direction.