Accurate and Efficient Multivariate Time Series Forecasting via Offline Clustering

Yiming Niu, Jinliang Deng, Lulu Zhang, Zimu Zhou, Yongxin Tong

Introduction

Predicting the future of complex systems—such as city-wide traffic flows, global weather patterns, or energy consumption—requires capturing "long-range dependencies." To make accurate predictions, a model must understand how a storm in one region affects another hours later, or how a morning rush hour in one neighborhood ripples across an entire metropolitan grid.

While Transformer architectures have revolutionized this field by capturing these dependencies, they come with a significant "Transformer Tax": a computational complexity that grows quadratically ($O(L^2)$) with the length of the input. This makes processing long historical windows prohibitively expensive for real-time applications. This paper introduces FOCUS (Forecaster with Offline Clustering Using Segments), a model that achieves state-of-the-art accuracy while slashing computational complexity to linear ($O(L)$).

The Challenge: Accuracy vs. Efficiency

Capturing long-range dependencies in multivariate time series (MTS) has traditionally been a balancing act between two gaps:

  1. The Accuracy Gap: Traditional models like CNNs or RNNs often struggle with very distant time steps, failing to capture the full context of the data.
  2. The Efficiency Gap: Previous attempts to optimize Transformers (e.g., Informer or Crossformer) often use sparse sampling or dimensionality reduction. While faster, these methods frequently "throw away" critical data, leading to a drop in forecasting precision.

FOCUS is built on the insight that real-world data is not random; it consists of recurring "events" (e.g., a specific traffic spike or a temperature fluctuation). Rather than comparing every segment of data to every other segment, FOCUS compares data to a set of "canonical" patterns, or prototypes.

Methodology

FOCUS operates in two distinct phases to decouple the complexity of dependency modeling from the input length.

1. Offline: Segment Clustering

The model first analyzes the entire training dataset, breaking it into small segments. It clusters these segments to identify prototypes—representative patterns that summarize high-level events.

Unlike standard clustering that relies on Euclidean distance (which only considers absolute values), FOCUS utilizes a composite loss function that includes Pearson correlation. This ensures the prototypes capture the morphology—the shape and trend—of the data, rather than just its magnitude. This allows the model to recognize a "morning rise" or a "nightly decline" as semantic events.

2. Online: Accelerated Dependency Modeling (ProtoAttn)

During the forecasting (online) phase, the model employs Prototypes Attentive Modeling (ProtoAttn). Instead of computing a massive $L \times L$ attention matrix, the model maps input segments to the fixed set of $k$ prototypes discovered offline. Because the number of prototypes ($k$) is a constant and significantly smaller than the input length ($L$), the computational complexity is reduced from quadratic to linear.

Dual-Branch Architecture

To handle the multivariate nature of the data (multiple sensors or entities), FOCUS utilizes a dual-branch structure:

  • Temporal Extractor: Models how patterns evolve over time within a single entity.
  • Entity Extractor: Models how different entities interact at the same point in time.
  • Parallel Fusion Module: A gating mechanism that dynamically weighs the importance of temporal versus entity features for the final prediction.

Experimental Results

The authors evaluated FOCUS on seven diverse, real-world multivariate benchmarks, including traffic (PEMS04, PEMS08), electricity (ETTh1, ETTm1), and global weather datasets. These benchmarks range in dimensionality from 7 to 862 separate variables.

  • Superior Accuracy: FOCUS achieved the top-1 accuracy in 26 out of 28 experimental settings, outperforming established models like PatchTST and Crossformer.
  • Efficiency Gains: In the PEMS08 dataset, FOCUS reduced FLOPs (floating-point operations) by nearly 50% compared to PatchTST while maintaining a lower peak memory footprint.
  • Robustness: By mapping data to learned prototypes, the model is significantly more resistant to random noise and outliers than models relying on raw point-to-point attention. It also demonstrated strong generalization when encountering "unseen" segments in test data.

Discussion and Future Work

FOCUS demonstrates that all-to-all attention is not a prerequisite for high-accuracy forecasting. By treating clusters as high-level system events, the model gains a form of semantic understanding that is both robust and efficient.

However, some challenges remain. The model's performance can be sensitive to hyperparameters like the number of prototypes ($k$) and segment length ($p$). Furthermore, extreme "regime shifts"—where the underlying physics of a system change entirely—might necessitate re-running the offline clustering phase. Future research may explore "zero-shot" forecasting, where prototypes learned in one domain (e.g., one city's traffic) are applied to predict patterns in an entirely different environment.

Summary updated

BibTeX
@inproceedings{niu2025accurate,
  author = {Yiming Niu and Jinliang Deng and Lulu Zhang and Zimu Zhou and Yongxin Tong},
  title = {{Accurate and Efficient Multivariate Time Series Forecasting via Offline Clustering}},
  booktitle = {ICDE},
  year = {2025},
  url = {https://openreview.net/forum?id=UQGZK9LyPP},
}
Figure