Multi-source Transfer Learning with Ensemble for Financial Time Series Forecasting

Qi-Qiao He, Patrick Cheong‐Iao Pang, Yain‐Whar Si

Abstract

Although transfer learning is proven to be effective in computer vision and natural language processing applications, it is rarely investigated in forecasting financial time series. Majority of existing works on transfer learning are based on single-source transfer learning due to the availability of open-access large-scale datasets. However, in the financial domain, the lengths of individual time series are relatively short and single-source transfer learning models are less effective. Therefore, in this paper, we investigate multi-source deep transfer learning for financial time series. We propose two multi-source transfer learning methods namely Weighted Average Ensemble for Transfer Learning (WAETL) and Tree-structured Parzen Estimator Ensemble Selection (TPEES). The effectiveness of our approach is evaluated on financial time series extracted from stock markets. Experiment results reveal that TPEES outperforms other baseline methods on the majority of multi-source transfer tasks.

Introduction

Deep learning models typically require massive datasets to perform effectively, a luxury often unavailable in financial time series forecasting. Historical price data for individual companies is often relatively "short," leading to overfitting when training complex models like LSTMs or MLPs. While single-source transfer learning (TL) can mitigate this, it often fails to capture the broader sectoral dynamics that influence a stock's price.

This paper addresses the challenge of leveraging multiple source stocks within the same industry to improve forecasting for a target stock. The primary hurdle in multi-source transfer is avoiding "negative transfer," where irrelevant or noisy data from a source stock degrades the target model's performance. To solve this, the authors propose two ensemble-based transfer learning methods: WAETL and TPEES.

Methodology

The proposed framework involves pre-training multiple deep learning models on different source datasets and then fine-tuning them on a specific target stock. Two novel ensembling strategies are introduced:

  1. Weighted Average Ensemble for Transfer Learning (WAETL): This method assigns weights to source models based on the similarity between the source and target time series. Similarity is measured using metrics such as Wasserstein Distance (WD) and Correlation Alignment (CORAL).
  2. Tree-structured Parzen Estimator Ensemble Selection (TPEES): This approach treats model selection as an optimization problem. It utilizes the Tree-structured Parzen Estimator (TPE) algorithm to automatically select and weight the best-performing models from a pool, effectively discarding those that contribute to negative transfer.
The TPEES workflow: from source data to an optimized ensembled model.

Experimental Setup

The methods were evaluated using Yahoo Finance data (2015–2019) across three industry groups:

  • G1 (Banking): Target stock ICBC; Source stocks include HSBC, HSB, and CCB.
  • G2 (Healthcare): Target stock JNJ; Source stocks include MRK and PFE.
  • G3 (Energy): Target stock PTR; Source stocks include CVX and TOT.

Models used a 22-day look-back window to predict the next day's closing price.

Results and Discussion

The experimental results demonstrate that multi-source transfer learning consistently outperforms standard non-transfer baselines like ARIMA and SVR, as well as single-source transfer models.

Comprehensive performance metrics (MAPE, RMSE, R²) for all industry groups.

As shown in the results, TPEES emerged as the superior method in most scenarios. For instance, in the Banking group (G1), the TPEES-LSTM model achieved a MAPE of 0.8888, a significant improvement over the standard LSTM (0.9499) and the ARIMA baseline (4.7079). The high $R^2$ values, often exceeding 0.94, indicate that the ensemble methods provide a much tighter fit to actual market movements. The success of TPEES suggests that optimization-based selection is more robust than fixed similarity metrics for identifying useful knowledge across multiple sources.

Conclusion

Multi-source transfer learning provides a powerful solution to the data scarcity and overfitting problems in financial forecasting. By intelligently ensembling models from related stocks, the TPEES method effectively captures sectoral trends while minimizing negative transfer. Future research will explore incorporating technical indicators and sentiment analysis to further enhance predictive accuracy.

Summary updated

Abstract

Although transfer learning is proven to be effective in computer vision and natural language processing applications, it is rarely investigated in forecasting financial time series. Majority of existing works on transfer learning are based on single-source transfer learning due to the availability of openaccess large-scale datasets. However, in financial domain, the lengths of individual time series are relatively short and singlesource transfer learning models are less effective. Therefore, in this paper, we investigate multi-source deep transfer learning for financial time series. We propose two multi-source transfer learning methods namely Weighted Average Ensemble for Transfer Learning (WAETL) and Tree-structured Parzen Estimator Ensemble Selection (TPEES). The effectiveness of our approach is evaluated on financial time series extracted from stock markets. Experiment results reveal that TPEES outperforms other baseline methods on majority of multi-source transfer tasks.
BibTeX
@inproceedings{he2020multi,
  author = {Qi-Qiao He and Patrick Cheong‐Iao Pang and Yain‐Whar Si},
  title = {{Multi-source Transfer Learning with Ensemble for Financial Time Series Forecasting}},
  booktitle = {WI/IAT},
  year = {2020},
  doi = {10.1109/wiiat50758.2020.00034},
}
Figure