R-Tuning: Wavelet-Decomposed Replay and Semantic Alignment for Continual Adaptation of Pretrained Time-Series Models

Tianyi Yin, Jingwei Wang, Chenze Wang, Han Wang, Jiexuan Cai, Min Liu, Yunlong Ma, Kun Gao, Yuting Song, Weiming Shen

Abstract

Foundation models for time-series forecasting offer powerful zero-shot capabilities but struggle with adaptation to shifting real-world distributions. Standard fine-tuning often leads to catastrophic forgetting of prior knowledge. We present R-Tuning, a framework that enables continual adaptation through frequency-aware synthetic data replay and latent semantic alignment. By leveraging wavelet transforms to generate diverse proxy samples and distillation to stabilize internal representations, R-Tuning significantly improves performance on new tasks while simultaneously enhancing retention on original distributions, all without requiring access to the original training data.

Introduction

The emergence of time-series foundation models like Chronos and Moirai has revolutionized forecasting. However, these models are typically static. In production environments, data distributions frequently shift due to changing consumer behaviors or hardware updates. Adapting these models via standard fine-tuning often results in "catastrophic forgetting," where the model loses its generalized capabilities. Furthermore, the original massive training datasets are often inaccessible due to privacy or scale, preventing traditional rehearsal methods. R-Tuning addresses this "learn fast, forget faster" dilemma by enabling effective adaptation using only the new task data and the frozen pre-trained weights.

Methodology

R-Tuning, or Replay Tuning, is built upon two core technical pillars designed to stabilize the adaptation process.

The R-Tuning framework: from wavelet-guided replay to semantic alignment.

Wavelet-guided Replay

To compensate for the lack of original training data, R-Tuning generates "proxy" samples. It passes random noise through the frozen, pre-trained model to produce synthetic signals. To enhance the utility of these samples, the framework employs a Redundant Wavelet Transform (RWT). This process decomposes synthetic signals into distinct frequency bands—separating low-frequency trends from high-frequency fluctuations. By selectively recombining these bands, R-Tuning creates a diverse set of frequency-aware data variants that more accurately reflect the underlying patterns of the original distribution than raw noise-based samples.

Semantic Alignment

To prevent internal representation drift during adaptation, R-Tuning utilizes a distillation mechanism termed Semantic Alignment. The adapting model is constrained to produce output distributions that remain semantically consistent with the original frozen model when processing synthetic replay samples. This acts as a stabilizer, ensuring the model retains its foundational "common sense" while specializing in the new task. The final objective function balances new task accuracy, distillation loss, and weight regularization.

Experimental Setup

The framework was evaluated across several domains and architectures:

  • Benchmarks: ETT (Energy), ECL (Electricity), Traffic, Weather, and Solar served as the "old" tasks.
  • New Task: A proprietary "Crowd Flow" dataset from a commercial area in Shanghai, featuring high-resolution pedestrian traffic data.
  • Architectures: Testing spanned six state-of-the-art models, including Chronos, Apollo (encoder-decoder), Moirai, AutoTimes (encoder-only), and TimerXL, GPT4TS (decoder-only).

Results and Discussion

R-Tuning demonstrates significant performance gains and robust knowledge retention.

Impact of replay ratio on MSE across various foundation models.

Performance and Retention

On new tasks, R-Tuning reduced Mean Absolute Error (MAE) and Mean Squared Error (MSE) by up to 46.9% and 46.8%, respectively. Crucially, while standard fine-tuning typically degrades performance on original tasks, R-Tuning achieved "positive knowledge transfer," improving old-task performance by up to 6.0% (MSE). This suggests that frequency-aware replay serves as a beneficial refresher for the model's general forecasting logic.

Efficiency and the 5% Sweet Spot

A key finding is the efficiency of the replay mechanism. As shown in the MSE analysis, near-optimal performance is achieved when synthetic replay data constitutes only 4%–5% of the new task's dataset size. This indicates that a massive replay buffer is not necessary to prevent forgetting, provided the replay samples are frequency-diverse.

Conclusion

R-Tuning provides a scalable solution for the continual adaptation of time-series foundation models. By focusing on frequency-domain characteristics through wavelet decomposition, it overcomes the limitations of data-free adaptation. Future work will explore online adaptation for streaming data and multi-modal extensions to incorporate auxiliary data types like text or imagery.

Summary updated

BibTeX
@inproceedings{yin2026r,
  author = {Tianyi Yin and Jingwei Wang and Chenze Wang and Han Wang and Jiexuan Cai and Min Liu and Yunlong Ma and Kun Gao and Yuting Song and Weiming Shen},
  title = {{R-Tuning: Wavelet-Decomposed Replay and Semantic Alignment for Continual Adaptation of Pretrained Time-Series Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i33.40010},
}
Figure