TL;DR
TimeFKA is a novel framework that enhances Time Series Foundation Models (TSFMs) by breaking down historical data into fine-grained trend and seasonal patterns stored in compact, reusable "codebooks." By allowing models to mix and match the best components from different contexts rather than retrieving noisy, whole-window samples, TimeFKA achieves significant accuracy gains across diverse datasets with minimal computational overhead.
What Problem Does This Paper Address?
Time Series Foundation Models (TSFMs) like Chronos or TimesFM have revolutionized forecasting by learning transferable representations from massive datasets. However, they often struggle with domain shift: the specific patterns in a new deployment (e.g., a specific cloud service's workload) might differ significantly from the general patterns seen during pre-training.
Existing solutions generally fall into two camps:
- Parametric Adaptation: Fine-tuning the model for every new domain, which is computationally expensive and hard to maintain at scale.
- Knowledge Augmentation (RAG): Retrieving similar historical windows to "remind" the model of relevant patterns.
The authors identify a critical flaw in current RAG approaches: they retrieve entire windows of data. A retrieved window might have a perfect trend match but a noisy seasonal match, or vice versa. This "coarse-grained" retrieval introduces context-specific noise that can actually degrade performance.
Methods
The authors propose TimeFKA (Fine-grained Knowledge Augmentation), a two-stage framework:
- Stage I: Shared Knowledge Consolidation: The goal here is to distill the vast pre-training corpus into a compact "pattern bank." The system decomposes time series windows into trend and seasonal components. It then uses a frozen TSFM encoder to generate embeddings, which are used to train Residual Reconstruction Blocks. These blocks learn hierarchical codebooks (similar to VQ-VAEs) that store prototypical trend and seasonal "atoms."
- Stage II: Task-Adaptive Augmentation: During downstream forecasting, the model doesn't search a massive database. Instead, it decomposes the current input and performs a multi-scale query against the compact codebooks learned in Stage I. A Hybrid Aggregation layer and a Gate Fusion module then intelligently blend these retrieved patterns with the TSFM's original internal representation to produce the final forecast.
Datasets
The authors evaluated TimeFKA on eight widely-used multivariate forecasting benchmarks:
- ETT (Electricity Transformer Temperature): Four subsets (ETTh1, ETTh2, ETTm1, ETTm2).
- Electricity: Power consumption of 321 clients.
- Exchange-rate: Daily exchange rates of eight countries.
- Traffic: Hourly road occupancy rates.
- Weather: 21 meteorological indicators.
They tested these across four representative TSFM backbones: Chronos-Bolt-base, TimesFM 2.5, Moirai 2, and TTM 2.
Key Insights and Ideas
- Decouple to Reuse: The core "aha" moment is that trend and seasonality are often independent physical processes. By decoupling them, the model can retrieve a trend pattern from one historical context and a seasonal pattern from another, creating a "recomposed" reference that is much more accurate than any single retrieved window.
- Residual Codebooks as Knowledge Compression: Instead of a standard RAG setup that requires storing and searching millions of raw data points, TimeFKA compresses this knowledge into small codebooks. This makes the system extremely efficient during inference.
- Universal Temporal Atoms: One of the most surprising findings is the transferability of the codebooks. Codebooks learned using one model (e.g., Chronos) can be used to improve the performance of a completely different architecture (e.g., TTM), suggesting that the learned "temporal atoms" capture fundamental properties of time series data.
Key Results
TimeFKA consistently outperformed both zero-shot TSFMs and state-of-the-art adaptation methods:
- Accuracy: It achieved average MSE reductions of 10.7% for Chronos-Bolt, 10.4% for TimesFM, 15.4% for Moirai, and a staggering 26.0% for TTM.
- Efficiency: Because it retrieves from compact codebooks rather than a raw database, TimeFKA is roughly 1.25x to 1.48x faster than existing RAG baselines like RAFT or TS-RAG.
- Robustness: The framework proved effective even when transferring codebooks across different model sizes (e.g., from a "Base" model to a "Tiny" model).
Limitations and Future Work
The authors acknowledge a few areas for improvement:
- Fixed Decomposition: The current model relies on a standard, fixed trend-seasonal decomposition. Moving toward a data-driven, learnable decomposition might capture more complex dynamics.
- Static Pattern Banks: Once the codebooks are learned in Stage I, they are frozen. In highly dynamic environments, the system could benefit from "continual learning" where the codebooks update as the model encounters new, unseen patterns in production.
Summary updated