TL;DR
SWIFT (Sample-Wise Instance Fine-Tuning) is a novel framework that specializes Time-Series Foundation Models (TSFMs) for a specific prediction window using only that individual sample’s own history. By shifting from "learning from others" to "learning from self" at pre-inference time, it achieves significant accuracy gains without requiring external datasets or future observations.
What Problem Does This Paper Address?
Time-Series Foundation Models (TSFMs) like Chronos and TimesFM have revolutionized forecasting with their zero-shot capabilities. However, real-world data is often non-stationary; local trends and seasonal behaviors shift over time.
While we can fine-tune these models to specific domains (few-shot adaptation), this typically requires a separate, curated dataset of similar sequences. In many practical scenarios—especially in streaming or privacy-sensitive environments—you don't have a "support dataset." You only have the history of the sequence you are currently trying to forecast.
Existing Test-Time Adaptation (TTA) methods try to solve this but often rely on progressively revealed future data or suffer from "parameter drift," where the model becomes unstable after repeated updates. SWIFT addresses this by treating a single sample's history as its own private training set, specializing the model just-in-time for the next prediction.
Methods
SWIFT introduces a "learning from self" paradigm that occurs entirely at pre-inference time. The framework consists of three core components:
- Self-Retrieval: To overcome the lack of data, SWIFT mines the target instance's own history. It uses a sliding window to find historical subsequences that are structurally similar to the current "look-back" window (the query). It employs Z-score normalization to focus on shape rather than scale and uses temperature-controlled sampling to ensure the "intrinsic support set" is diverse.
- Frequency-Aware Loss (FreAL): Fine-tuning on a tiny amount of data risks overfitting to local noise. SWIFT mitigates this by calculating loss in the frequency domain. It identifies the most critical frequency components (global dynamics) and assigns them higher weights while suppressing high-frequency noise. It also includes an anchor loss to ensure the adapted model doesn't drift too far from the robust foundation model.
- Deviation-Aware Dynamic Fusion: This acts as an "adaptive safety valve." During inference, SWIFT generates predictions from both the original zero-shot model and the newly adapted expert. If the two predictions deviate significantly—suggesting potential over-adaptation or instability—the system automatically increases the weight of the original foundation model.
Datasets
The authors evaluated SWIFT across four major TSFM backbones (Chronos-2, TimesFM-2.5, Moirai-2, and TiRex) using three benchmark collections:
- Chronos Benchmark II: Specifically six long-history tasks (Traffic, Electricity, Exchange, ETTm, ETTh, and ERCOT).
- fev-bench: 46 diverse real-world forecasting tasks.
- GIFT-Eval: 24 tasks across varying domains and sampling frequencies.
To ensure the model had enough "self" to learn from, the evaluation was restricted to tasks with a history length of at least 2048 time steps.
Key Insights and Ideas
- The History is the Dataset: The most profound shift is the realization that a sufficiently long time series contains recurrent motifs that can serve as high-quality supervision for its own future.
- Spectral Consistency over Point Accuracy: In low-data regimes, matching the "rhythm" (frequency) of the past is often more robust than trying to minimize point-wise error, which can lead to chasing noise.
- Transparency for the End-User: Because SWIFT only needs the input sample, the fine-tuning process is "transparent." A user simply requests a forecast, and the model specializes itself behind the scenes without needing a manual training phase.
Key Results
SWIFT consistently outperformed zero-shot, few-shot, and TTA baselines:
- Accuracy Gains: On Chronos Benchmark II, it achieved an average reduction of 6.41% in MASE (point accuracy) and 4.83% in WQL (probabilistic accuracy).
- Win Rates: Compared to zero-shot inference, SWIFT improved MASE in 87.5% of configurations and WQL in 95.8% of configurations.
- Efficiency: Despite performing 15 LoRA update steps per sample, SWIFT’s average execution time was roughly 6.8 seconds—significantly faster than standard dataset-level fine-tuning and comparable to lightweight TTA methods.
Limitations and Future Work
- History Requirements: SWIFT requires a relatively long observed history ($T \ge 2048$) to find meaningful patterns. It may not be suitable for very short or entirely novel sequences.
- Computational Overhead: While 6.8 seconds is fast for a researcher, it adds latency compared to pure zero-shot inference. The authors note that future work will focus on reducing this computational cost.
- Hyperparameter Sensitivity: The effectiveness of the spectral weighting and the fusion "safety valve" relies on thresholds ($\tau_{spec}, \tau_{fus}$) that might need tuning for vastly different data types.
Summary updated