A Pattern Discovery Approach to Multivariate Time Series Forecasting

Yunyao Cheng, Chenjuan Guo, Kaixuan Chen, Kai Zhao, Bin Yang, Jiandong Xie, Christian S. Jensen, Feiteng Huang, Kai ZhengarXiv 2022 · arXiv · OpenReview

Abstract

Multivariate time series forecasting constitutes important functionality in cyber-physical systems, whose prediction accuracy can be improved significantly by capturing temporal and multivariate correlations among multiple time series. State-of-the-art deep learning methods fail to construct models for full time series because model complexity grows exponentially with time series length. Rather, these methods construct local temporal and multivariate correlations within subsequences, but fail to capture correlations among subsequences, which significantly affect their forecasting accuracy. To capture the temporal and multivariate correlations among subsequences, we design a pattern discovery model, that constructs correlations via diverse pattern functions. While the traditional pattern discovery method uses shared and fixed pattern functions that ignore the diversity across time series. We propose a novel pattern discovery method that can automatically capture diverse and complex time series patterns. We also propose a learnable correlation matrix, that enables the model to capture distinct correlations among multiple time series. Extensive experiments show that our model achieves state-of-the-art prediction accuracy.

Introduction: The Few-Shot Challenge

Time series forecasting is ubiquitous, but in fields like biology or physics, gathering high-quality data is often expensive or physically limited. This leads to the "few-shot" problem: training a model to predict the future accurately with only a handful of training samples.

Existing sequential Latent Variable Models (LVMs) typically rely on Markovian transitions—predicting the next state based only on the current one. While this works for short-term motion, it struggles to capture "global" patterns like multi-month seasonality or long-term trends. Furthermore, many meta-learning models act as "black boxes," making it difficult for researchers to interpret the patterns the model has actually learned.

To address these limitations, we introduce MetaGP, a framework that shifts the forecasting task into a Gaussian Process (GP) paradigm. By using a GP as the dynamic function, the model maintains strong correlations between observations even if they are far apart in time, effectively solving the long-term dependency issue.

Comparison of Sequential LVM (a) and the proposed Meta-learning LVM (b) architectures.

The core innovation lies in three components:

  1. Attention-based Gaussian Location Learning: Instead of using raw timestamps, the model uses "patch attention" to encode historical subsequences into latent locations. This allows the model to focus on the most salient temporal features.
  2. Kernel Association Search (KAS): Traditional GPs use a fixed kernel. MetaGP uses a 2-level search strategy to automatically combine basic kernels (Trend, Seasonal, Short-term, Long-term) into a complex, analytic kernel function. By using weighted edges and one-shot selection, KAS reduces search complexity from $O(|K|^R)$ to $O(|K|R)$, making it feasible for deep learning pipelines.
  3. Multivariate Covariance Learning: To handle multiple variables, the model learns a cross-variable weight matrix. This allows it to capture how different time series (e.g., different microbial species) correlate with one another.
Examples of time series patterns, including trends and multi-scale seasonality, which the KAS component identifies.

Experimental Evaluation

The model was evaluated on seven benchmark datasets across physical and biological domains, including Gravity-16 (image sequences of falling objects) and MC (Microbial Community), a real-world biological dataset depicting changes in microbial community sizes over time.

Key Results

  • State-of-the-Art Accuracy: On the Gravity-16 dataset, MetaGP achieved a distance error (Dist) of 2.06, significantly lower than the next best meta-learning model, meta-GRU-res (2.88).
  • Superiority in Biology: On the real-world MC dataset, MetaGP outperformed all baselines in Mean Absolute Error (MAE) and Mean Squared Error (MSE), proving its utility in noisy scientific domains.
  • Long-term Robustness: Ablation studies showed that as the forecasting horizon increased from 12 to 48 steps, MetaGP’s performance remained stable compared to models without the GP component, which saw their errors explode.

Conclusion

MetaGP demonstrates that in few-shot scenarios, a well-constructed "prior" (the GP kernel) is more effective than attempting to boost likelihood with massive amounts of data. By making meta-knowledge explicit through searchable analytic kernels, the model provides both high accuracy and interpretability for complex multivariate time series.

Summary updated

Abstract

Multivariate time series forecasting constitutes important functionality in cyber-physical systems, whose prediction accuracy can be improved significantly by capturing temporal and multivariate correlations among multiple time series. State-of-the-art deep learning methods fail to construct models for full time series because model complexity grows exponentially with time series length. Rather, these methods construct local temporal and multivariate correlations within subsequences, but fail to capture correlations among subsequences, which significantly affect their forecasting accuracy. To capture the temporal and multivariate correlations among subsequences, we design a pattern discovery model, that constructs correlations via diverse pattern functions. While the traditional pattern discovery method uses shared and fixed pattern functions that ignore the diversity across time series. We propose a novel pattern discovery method that can automatically capture diverse and complex time series patterns. We also propose a learnable correlation matrix, that enables the model to capture distinct correlations among multiple time series. Extensive experiments show that our model achieves state-of-the-art prediction accuracy.
BibTeX
@misc{cheng2022pattern,
  author = {Yunyao Cheng and Chenjuan Guo and Kaixuan Chen and Kai Zhao and Bin Yang and Jiandong Xie and Christian S. Jensen and Feiteng Huang and Kai Zheng},
  title = {{A Pattern Discovery Approach to Multivariate Time Series Forecasting}},
  year = {2022},
  eprint = {2212.10306},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2212.10306},
}
Figure