Introduction
Deep learning has significantly advanced time series forecasting, yet these models remain heavily dependent on large-scale datasets. In many practical applications—such as monitoring new industrial facilities or tracking emerging public health crises—historical data is often insufficient to train complex architectures like Transformers or RNNs from scratch. While transfer learning has been highly successful in domains like Computer Vision and Natural Language Processing, it frequently encounters significant hurdles in time series due to "Dataset Shift." This phenomenon occurs when the underlying temporal dynamics and variable relationships differ drastically between the source and target domains, often resulting in negative transfer, where the source data actually degrades model performance.
The SeDAN Framework
To address these challenges, we present the Sequence Decomposition Adaptation Network (SeDAN). Unlike traditional "one-size-fits-all" adaptation methods that attempt to align entire data distributions, SeDAN recognizes that time series are composed of distinct components with varying degrees of transferability.
The core philosophy of SeDAN is to decouple complex temporal features into seasonal and trend components before applying specialized adaptation strategies to each.
Methodology
SeDAN's architecture is built upon three primary technical pillars:
- Implicit Contrastive Decomposition (ICD): Rather than relying on rigid mathematical decompositions (e.g., STL), SeDAN employs self-supervised contrastive learning to learn an implicit split between seasonal and trend features. This process is supported by a memory bank and an Online Prototype Update (OPU) mechanism, ensuring the decomposition remains robust across diverse examples.
- Seasonal Feature Adaptation: Seasonal patterns tend to be periodic and more "universal" across domains. SeDAN utilizes Joint Maximum Mean Discrepancy (JMMD) to align these distributions, leveraging the assumption that the rhythmic nature of data is a highly transferable form of knowledge.
- Optimal Local Adaptation (OLA) for Trends: Trends are typically domain-specific and volatile. SeDAN treats trend adaptation as a global optimal matching problem. By using a differentiable convex optimization layer, the model identifies and aligns similar subsequences between domains, regardless of when they occur. This ensures that only "temporal-invariant" knowledge is transferred.
Experimental Evaluation
SeDAN was rigorously evaluated across five multivariate time series benchmarks, including electricity transformer temperatures (ETT), local climate data (WTH), exchange rates (EXC), and influenza-like illness (ILI) statistics. These datasets represent a wide variety of scales and frequencies.
Key Results
The experimental results demonstrate that SeDAN consistently outperforms both single-domain and cross-domain baselines:
- Performance Gains: Compared to models trained exclusively on target data (e.g., Informer, TCN), SeDAN achieved an average reduction of 27.0% in Mean Squared Error (MSE) and 16.5% in Mean Absolute Error (MAE).
- Superiority over Standard Transfer: SeDAN outperformed conventional transfer methods like DAN and JAN by an average of 10.9% in MSE.
- Mitigating Negative Transfer: While standard fine-tuning and adaptation methods occasionally performed worse than vanilla models, SeDAN maintained stability and improved performance across all tested source-target pairs.
Conclusion
SeDAN demonstrates that the key to effective time series transfer learning lies in decomposition. By separating stable seasonal patterns from volatile trends and applying tailored adaptation techniques, the model effectively mitigates the risks of dataset shift. Future research will focus on developing automated metrics to determine the optimal "when" for transfer and optimizing the computational efficiency of the OLA module for long-sequence applications.
Summary updated