TimeRAG: Boosting LLM Time Series Forecasting via Retrieval-Augmented Generation

Silin Yang, Dong Wang, Haoqi Zheng, Ruochun Jin

TL;DR

TimeRAG is the first framework to apply Retrieval-Augmented Generation (RAG) to time series forecasting with Large Language Models (LLMs). By retrieving similar historical patterns using Dynamic Time Warping and feeding them as context to a frozen LLM, it achieves a 2.97% average accuracy boost across diverse domains without requiring massive retraining.

What Problem Does This Paper Address?

While Large Language Models (LLMs) have revolutionized NLP, applying them to time series forecasting remains challenging. Current LLM-based forecasting solutions often require expensive fine-tuning and struggle to transfer across different domains (e.g., moving from financial data to weather data).

Furthermore, LLMs are prone to "hallucinations"—generating outliers or fabricated patterns that don't align with actual historical trends. Because standard LLMs lack a mechanism to "look up" similar historical examples during inference, they rely entirely on their internal weights, which may not capture the specific nuances of a new dataset.

Methods

The authors propose TimeRAG, a framework that enhances LLMs by providing them with relevant historical context. The process consists of three main stages:

  1. Knowledge Base Construction: Instead of storing every raw data point, the authors use a sliding window to slice historical sequences into segments. They then apply K-means clustering to these segments and store only the cluster centroids. This creates a compact, representative "knowledge base" of common waveforms and trends.
  2. Pattern Retrieval: When a new query (a sequence to be forecasted) arrives, the system searches the knowledge base for the most similar patterns. Crucially, they use Dynamic Time Warping (DTW) as the distance metric. Unlike standard Euclidean distance, DTW is "shape-aware" and tolerant of temporal distortions (e.g., a pattern that is slightly stretched or shifted in time).
  3. Forecasting via Prompting: The retrieved "referential sequences" are combined with the input query into a natural language prompt. This prompt is fed into an LLM (the authors used a Llama3-based model with a "reprogramming layer" to align numerical data with text tokens). The LLM then uses these examples as a guide to predict the next steps.

Datasets

The framework was evaluated on the M4 dataset, a massive benchmark containing 100,000 time series from various domains (finance, demographics, etc.) across six different frequencies:

  • Yearly, Quarterly, Monthly, Weekly, Daily, and Hourly.

This variety allowed the authors to test how well TimeRAG adapts to different temporal granularities and domain-specific patterns.

Key Insights and Ideas

  • Context is King: The "aha moment" here is that LLMs perform significantly better at forecasting when they are shown "what similar situations looked like in the past." This mirrors how RAG works in text (providing a document to answer a question).
  • DTW > Euclidean Distance: For time series, matching exact values is less important than matching the shape of the wave. Using DTW allows the model to find relevant context even if the timing isn't a perfect 1:1 match.
  • Efficiency through Clustering: By using K-means to build the knowledge base, the authors avoid the computational nightmare of searching through millions of raw historical windows, making the retrieval step feasible for real-time applications.
  • Zero-Parameter Enhancement: TimeRAG improves performance without modifying the foundational parameters of the underlying LLM, making it a "plug-and-play" enhancement for existing models like Time-LLM.

Key Results

  • Accuracy Boost: TimeRAG improved the prediction accuracy of the baseline model (Time-LLM) by 2.97% on average.
  • Domain Success: The most significant gains were seen in the "Weekly" frequency data, where the model achieved a 13.12% improvement in MASE (Mean Absolute Scaled Error).
  • SOTA Performance: TimeRAG outperformed several state-of-the-art specialized models, including iTransformer, FEDformer, and DLinear. In 18 different comparative analyses (across various metrics and frequencies), TimeRAG secured a "top-three" position 14 times.

Limitations and Future Work

While the paper demonstrates strong results, a few areas remain for exploration:

  • Computational Overhead: Although clustering helps, the retrieval step (calculating DTW for multiple candidates) still adds latency compared to a direct forward pass of a standard model.
  • Prompt Sensitivity: The paper uses a specific natural language template; future work could explore how different prompt engineering strategies or "chain-of-thought" reasoning might further reduce hallucinations.
  • Scaling the KB: The authors suggest that future research should look into more advanced "knowledge-enhanced sequential data management" to handle even larger, more complex knowledge bases.

Summary updated

Abstract

Although the rise of large language models (LLMs) has introduced new opportunities for time series forecasting, existing LLM-based solutions require excessive training and exhibit limited transferability. In view of these challenges, we propose TimeRAG, a framework that incorporates Retrieval-Augmented Generation (RAG) into time series forecasting LLMs, which constructs a time series knowledge base from historical sequences, retrieves reference sequences from the knowledge base that exhibit similar patterns to the query sequence measured by Dynamic Time Warping (DTW), and combines these reference sequences and the prediction query as a textual prompt to the time series forecasting LLM. Experiments on datasets from various domains show that the integration of RAG improved the prediction accuracy of the original model by 2.97% on average.
BibTeX
@inproceedings{yang2025timerag,
  author = {Silin Yang and Dong Wang and Haoqi Zheng and Ruochun Jin},
  title = {{TimeRAG: Boosting LLM Time Series Forecasting via Retrieval-Augmented Generation}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=6s603S7S1Z},
}

Loading related content...

Figure