Tokenizing Stock Prices for Enhanced Multi-Step Forecast and Prediction

Qiang (Vincent) Qu, Zhuohang Zhu, Haodong Chen, Qiang Qu 0004, Xiaoming Chen, Vera ChungarXiv 2025 · DOI · arXiv

Abstract

Financial time series forecasting remains a significant challenge due to the non-stationary nature of market data and the inherent noise in daily price movements. This paper discusses the Patched Channel Integration Encoder (PCIE), a novel architecture that applies tokenization principles—similar to those used in Large Language Models—to stock price segments. By utilizing Univariate Patching and an Adaptive Temporal Learning (ATL) block, PCIE effectively captures both price levels and price changes. Experimental results on US_71 and US_14L datasets demonstrate that PCIE outperforms existing state-of-the-art models in multi-step forecasting horizons, providing a more robust framework for financial market analysis.

1. Introduction

Predicting stock market trends is a complex task characterized by high volatility and non-stationary data, where statistical properties such as mean and variance shift over time. Traditional research typically bifurcates into two distinct tasks: forecasting absolute price levels (often following a log-normal distribution) and predicting percentage changes or returns (typically following a normal distribution). Models optimized for one task frequently underperform on the other.

Furthermore, multi-step forecasting—predicting prices over several future days—is often plagued by the "snowball effect." Iterative models that use their own previous predictions as inputs for subsequent steps tend to accumulate errors rapidly. To address these challenges, the Patched Channel Integration Encoder (PCIE) introduces a method to process time series data as discrete "patches" or tokens, allowing for more stable and accurate long-term predictions.

2. Methodology

The PCIE architecture is designed to handle the multi-dimensional and noisy nature of financial data through several key components:

2.1 Univariate Patching (Tokenization)

Rather than analyzing individual daily data points, PCIE breaks the time series into overlapping segments known as patches. This approach is analogous to tokenization in natural language processing. By treating a sequence of prices as a single token, the model can ignore localized stochastic noise and focus on broader temporal patterns, which significantly enhances the signal-to-noise ratio and reduces computational complexity.

2.2 Adaptive Temporal Learning (ATL) Block

Recognizing that different stocks exhibit unique behaviors, the ATL block dynamically determines the optimal method for embedding patches. It utilizes a combination of linear layers and Multi-Layer Perceptrons (MLPs) to generate the most effective latent representations for a specific dataset, ensuring the model remains flexible across different market sectors.

2.3 Channel Mixing and Self-Attention

Financial data consists of multiple channels, including Open, High, Low, Close, and Volume (OHLCV). PCIE integrates these channels along with their corresponding percentage changes. A modified Transformer-style self-attention mechanism is employed to learn the complex correlations between these channels, capturing market dynamics that univariate models often overlook.

2.4 Direct Multi-Step Output

To mitigate the error accumulation found in iterative forecasting, PCIE employs a flatten layer followed by a linear head. This structure allows the model to predict the entire future sequence (e.g., a 60-day horizon) in a single computational step, ensuring greater stability over longer periods.

3. Experimental Setup

The model's performance was validated using two comprehensive real-world datasets:

  • US_71: This dataset includes historical data for 71 high-volume U.S. stocks across nine major industries from 2016 to 2023.
  • US_14L: A long-term dataset featuring 14 major stocks spanning nearly 20 years (2005–2023). This dataset specifically tests the model’s resilience against significant long-term distribution shifts and various economic cycles.

4. Results and Discussion

PCIE was benchmarked against several state-of-the-art (SOTA) models, including PatchTST, Informer, Autoformer, and D-Va.

4.1 Comparative Performance

PCIE consistently achieved the lowest Mean Squared Error (MSE) and Mean Absolute Error (MAE) across all tested forecasting horizons (10, 20, 40, and 60 days). While models like Autoformer and D-Va showed competence in predicting price changes, they struggled with absolute price levels. PCIE demonstrated superior generalization by accurately handling both metrics.

4.2 The Role of Hybrid Inputs

A significant finding was the impact of hybrid inputs. By combining absolute prices ($p$) and price changes ($\Delta p$), all tested models saw a performance boost. This suggests that providing both "scale" (where the price is) and "momentum" (where the price is going) is a critical factor for financial AI success.

4.3 Ablation Studies

Ablation studies confirmed that the patching/tokenization process is the most vital component of the PCIE architecture. Removing this feature led to a significant increase in error rates, proving that grouping time steps is essential for filtering market noise and reducing the quadratic computational costs associated with standard Transformer models.

5. Conclusion and Future Work

The Patched Channel Integration Encoder represents a significant advancement in technical analysis for financial markets. By treating price segments as tokens and integrating cross-channel correlations, it provides a robust solution for multi-step forecasting.

However, the model currently relies solely on quantitative price and volume data. Future research will explore the integration of fundamental analysis, such as macroeconomic indicators and sentiment analysis from news and social media. Additionally, implementing online learning capabilities would allow the model to adapt to real-time market shifts without requiring full periodic retraining.

Summary updated

BibTeX
@misc{qu2025tokenizing,
  author = {Qiang (Vincent) Qu and Zhuohang Zhu and Haodong Chen and Qiang Qu and Xiaoming Chen and Vera Chung},
  title = {{Tokenizing Stock Prices for Enhanced Multi-Step Forecast and Prediction}},
  year = {2025},
  eprint = {2504.17313},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.17313},
}
Figure