Self-Adaptive Scale Handling for Forecasting Time Series with Scale Heterogeneity

Xu Zhang, Zhengang Huang, Yunzhi Wu, Xun Lu, Erpeng Qi, Yunkai Chen, Zhongya Xue, Peng Wang, Wei Wang

TL;DR

This paper introduces a novel "Self-Adaptive Scale-handling" (AS) module to tackle the challenge of forecasting time series with vastly different numerical scales (scale heterogeneity). By learning adaptive scaling factors for each time series, AS preserves crucial semantic information and significantly improves forecasting accuracy compared to traditional methods.

What Problem Does This Paper Address?

Time series forecasting (TSF) is crucial for many applications, but existing methods often assume scale-homogeneous data, where all time series have similar numerical ranges. In real-world industrial scenarios, like financial product sales or e-commerce item sales, time series can differ by orders of magnitude – a phenomenon called scale heterogeneity. While these series might share similar temporal patterns, jointly modeling them is desirable for better data utilization. However, standard scaling techniques like global normalization compress low-scale signals, and window-based scaling destroys semantic discriminability and amplifies errors during inverse scaling. This paper addresses the critical need for a method that can effectively handle these scale differences without sacrificing important information or introducing new problems.

Methods

The core contribution is the Self-Adaptive Scale-handling (AS) module, designed to be seamlessly integrated into existing TSF models. AS comprises two key sub-modules:

  • Scale Calibrating (SC) Sub-module: This module learns adaptive scale factors tailored to each input time series. It first extracts temporal features from the input window using convolutional blocks. Then, a fully connected (FC) block, including an MLP network and a sigmoid function, learns a calibration coefficient ($\lambda_i$). This coefficient is used to adjust a "prior mean scaling factor" ($v_i$), which is derived from the input data itself. The goal is to create a calibrated scale factor ($\hat{v}_i = \lambda_i \times v_i$) that better reflects the true scale of the data while preserving its semantic meaning.
  • Scaling Selection (SS) Sub-module: To prevent "over-calibration" – where the model might unnecessarily adjust already well-performing scale factors, leading to performance degradation – the SS sub-module is introduced. It uses a parameterized Bernoulli distribution, implemented with the Gumbel-Softmax trick for differentiability, to decide whether to use the calibrated scale factor ($\hat{v}_i$) or the original prior mean scaling factor ($v_i$). This allows the model to autonomously select the best scaling strategy for each input window.

The overall framework involves extracting features, computing calibration coefficients and flags, and then applying the selected scaling to the prior mean scaling factor before feeding it into the forecasting model.

Datasets

The paper uses real-world industrial datasets collected from Ant Fortune, a financial platform. These datasets are divided into two groups:

  • Fund₁: 66 fund sales datasets.
  • Fund₂: 106 fund sales datasets.

These datasets are suitable for validating the method because they exhibit significant scale heterogeneity across different fund products. The data is split into training, validation, and testing sets with a ratio of 7:1:2.

Key Insights and Ideas

  • Scale heterogeneity is a significant challenge in real-world TSF: The paper highlights that many industrial applications face this issue, and existing methods are ill-equipped to handle it.
  • Preserving semantic discriminability is crucial: Unlike traditional methods that compress or distort scale information, AS aims to retain the inherent differences in magnitude between time series, which is vital for accurate forecasting.
  • Adaptive scaling is more effective than uniform scaling: By learning scale factors specific to each input, AS avoids the pitfalls of global or window-based normalization, leading to better signal-to-noise ratios and reduced reconstruction errors.
  • Preventing over-calibration is key: The SS sub-module is a clever addition that ensures the model doesn't harm performance by over-adjusting scale factors, demonstrating the importance of selective adaptation.
  • WMAPE loss is better suited for scale-heterogeneous TSF: The study found that Weighted Mean Absolute Percentage Error (WMAPE) is a more robust loss function than Mean Squared Error (MSE) for this type of data, as MSE is sensitive to scale differences and can lead to unstable training.

Key Results

  • Consistent performance improvements: When integrated with popular TSF models like Informer, Transformer, Performer, and Autoformer, the AS module consistently improved their performance across both Fund₁ and Fund₂ datasets.
  • AS outperforms baselines: AS achieved the best results compared to models with no preprocessing, vanilla scaling (VS), global standardization (std), and normalization (nor).
  • Ablation study confirms effectiveness: The SC sub-module alone showed improvements, but the full AS module (SC + SS) consistently outperformed vanilla scaling, especially in scenarios where SC alone might over-calibrate.
  • WMAPE vs. MSE: Using WMAPE as a training loss yielded better results than MSE for scale-heterogeneous forecasting.

Limitations and Future Work

The authors acknowledge a few limitations:

  • Limited variables per product: The current experiments focus on two variables (purchase and redemption volumes) per product. Extending to more variables with greater scale diversity within a single product might lead to less stable calibration factors.
  • Instability of per-sample calibration: While the batch-mean strategy partially mitigates this, per-sample calibration factors can still exhibit instability during training.

Future work suggested includes:

  • Exploring more robust adaptive factor estimation methods.
  • Investigating cross-variable scale coordination for more complex scenarios.
  • Extending the AS module to broader industrial applications with scale heterogeneity.

Summary updated

Abstract

Current time series forecasting (TSF) research predominantly focuses on scale-homogeneous data, where different time series share similar numerical magnitude ranges. However, in real-world industrial scenarios such as financial product sales, different time series often differ by orders of magnitude (scale heterogeneity). Since these series share similar temporal patterns, joint modeling is desirable for better data utilization, yet existing scaling methods either compress low-scale signals (global normalization) or destroy semantic discriminability and amplify inverse-scaling errors (window-based scaling). This paper proposes a self-Adaptive Scale-handling (AS) module that learns adaptive scale factors tailored to each input, preserving semantic discriminability while reducing inverse-scaling errors. AS consists of Scale Calibrating (SC), which calibrates prior mean scaling factors through neural networks, and Scaling Selection (SS), which decides whether to apply calibration or retain the original factor, avoiding over-calibration. Experiments on real-world fund sales datasets from Ant Fortune and Alipay show that AS seamlessly integrates into popular TSF models and consistently improves their performance. The code and dataset are available at the link https://github.com/Meteor-Stars/ASTSF.
BibTeX
@inproceedings{zhang2024self,
  author = {Xu Zhang and Zhengang Huang and Yunzhi Wu and Xun Lu and Erpeng Qi and Yunkai Chen and Zhongya Xue and Peng Wang and Wei Wang},
  title = {{Self-Adaptive Scale Handling for Forecasting Time Series with Scale Heterogeneity}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  doi = {10.1109/icassp48485.2024.10446923},
}
Figure