Multi-source Transfer Learning with Ensemble for Financial Time Series Forecasting

Qi-Qiao He, Patrick Cheong-Iao Pang, Yain-Whar Si

TL;DR

This paper introduces two novel ensemble-based methods—WAETL and TPEES—to improve financial time series forecasting by leveraging data from multiple related stocks. By using multi-source transfer learning, the authors successfully mitigate the overfitting problems common in "short" financial datasets, with the optimization-based TPEES method emerging as the top performer.

What Problem Does This Paper Address?

In fields like Computer Vision (CV) or Natural Language Processing (NLP), deep learning models thrive on massive datasets. However, in the financial domain, the historical price data for a single company is relatively "short." Training a complex model on such limited data often leads to overfitting, where the model memorizes noise rather than learning useful patterns.

While Transfer Learning (TL)—taking a model trained on one task and fine-tuning it for another—is a standard solution, most financial research focuses on single-source transfer (e.g., using only one other stock's data). This is often insufficient. The authors argue that a stock's price is influenced by its entire sector (e.g., HSBC is correlated with other banks). The challenge lies in how to effectively combine knowledge from multiple source stocks without suffering from "negative transfer," where irrelevant data actually makes the forecast worse.

Methods

The authors propose a framework where multiple deep learning models (LSTMs and MLPs) are pre-trained on different source datasets and then fine-tuned on a specific target stock. They introduce two ways to ensemble these models:

  1. Weighted Average Ensemble for Transfer Learning (WAETL): This method assigns a weight to each source model based on how "similar" the source stock's time series is to the target stock's. They tested several distance functions to measure this similarity, including:
  2. CORAL loss (Correlation Alignment)
    • Wasserstein Distance (WD)
    • Dynamic Time Warping (DTW)
    • Pearson Correlation Coefficient (PCC)
  3. Tree-structured Parzen Estimator Ensemble Selection (TPEES): Instead of relying on pre-defined similarity metrics, TPEES treats model selection as an optimization problem. It uses the Tree-structured Parzen Estimator (TPE)—a popular algorithm for hyperparameter tuning—to decide which models from the "pool" should be included in the final ensemble and how much they should be weighted. Crucially, TPEES can choose to completely ignore models that perform poorly.

Datasets

The researchers evaluated their methods using stock data from Yahoo Finance spanning 2015 to 2019, organized into three industry groups:

  • G1 (Banking): Source stocks like HSBC, HSB, and CCB; Target stock: ICBC.
  • G2 (Healthcare): Source stocks like Merck (MRK) and Pfizer (PFE); Target stock: Johnson & Johnson (JNJ).
  • G3 (Energy): Source stocks like Chevron (CVX) and Total (TOT); Target stock: PetroChina (PTR).

The data was preprocessed using a 22-day look-back window to predict the price for the following day.

Key Insights and Ideas

  • The "Short Data" Solution: Multi-source transfer learning consistently outperforms training on a single stock's data. By pulling in "knowledge" from an entire industry, the models become more robust.
  • Optimization Beats Intuition: TPEES generally outperformed WAETL. This suggests that mathematical similarity (like DTW or Correlation) doesn't always capture the predictive value of a source dataset. TPEES’s ability to simply discard "bad" models makes it more resilient to negative transfer.
  • Distance Metrics Matter: For the WAETL method, Wasserstein Distance (WD) and CORAL were found to be the most stable and robust distance functions for calculating weights, outperforming the more traditional Pearson Correlation in many scenarios.
  • Industry Correlation is Key: The success of these methods confirms that stocks within the same sector share underlying features that deep learning models can exploit.

Key Results

The authors compared their methods against baselines like ARIMA, Support Vector Regression (SVR), and standard LSTMs/MLPs without transfer learning.

  • TPEES Dominance: TPEES achieved the best results (lowest MAPE and RMSE, highest $R^2$) in the majority of multi-source transfer tasks.
  • Error Reduction: In Group 1 (Banking), the TPEES-based LSTM model achieved a MAPE of 0.8888, significantly better than the standard LSTM (0.9499) or the traditional ARIMA model (4.7079).
  • Better Fitting: The $R^2$ values for the proposed ensemble methods were consistently higher, often exceeding 0.94, indicating a very high degree of fit to the actual stock price movements compared to non-transfer methods.

Limitations and Future Work

  • Negative Correlation: The current study focuses on positive correlations within industries. The authors note that incorporating stocks with negative correlations might provide additional hedging-style insights.
  • Feature Set: The models currently rely primarily on historical price data. Future work could include technical indicators (RSI, MACD) or even sentiment analysis from news data.
  • Forecast Horizon: The experiments focused on 1-day-ahead forecasting. Extending this to multi-step-ahead forecasting remains a challenge for future research.
  • Computational Cost: Ensembling multiple deep learning models and running TPE optimization is computationally expensive compared to simpler statistical models.

Summary updated

Abstract

Although transfer learning is proven to be effective in computer vision and natural language processing applications, it is rarely investigated in forecasting financial time series. Majority of existing works on transfer learning are based on single-source transfer learning due to the availability of openaccess large-scale datasets. However, in financial domain, the lengths of individual time series are relatively short and singlesource transfer learning models are less effective. Therefore, in this paper, we investigate multi-source deep transfer learning for financial time series. We propose two multi-source transfer learning methods namely Weighted Average Ensemble for Transfer Learning (WAETL) and Tree-structured Parzen Estimator Ensemble Selection (TPEES). The effectiveness of our approach is evaluated on financial time series extracted from stock markets. Experiment results reveal that TPEES outperforms other baseline methods on majority of multi-source transfer tasks.
BibTeX
@inproceedings{he2020multi,
  author = {Qi-Qiao He and Patrick Cheong-Iao Pang and Yain-Whar Si},
  title = {{Multi-source Transfer Learning with Ensemble for Financial Time Series Forecasting}},
  booktitle = {WI/IAT},
  year = {2020},
  doi = {10.1109/wiiat50758.2020.00034},
}
Figure