Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models

Abstract

Time series analysis is crucial for understanding dynamics of complex systems. Recent advances in foundation models have led to task-agnostic Time Series Foundation Models (TSFMs) and Large Language Model-based Time Series Models (TSLLMs), enabling generalized learning and integrating contextual information. However, their success depends on large, diverse, and high-quality datasets, which are challenging to build due to regulatory, diversity, quality, and quantity constraints. Synthetic data emerge as a viable solution, addressing these challenges by offering scalable, unbiased, and high-quality alternatives. This survey provides a comprehensive review of synthetic data for TSFMs and TSLLMs, analyzing data generation strategies, their role in model pretraining, fine-tuning, and evaluation, and identifying future research directions.

Introduction

The "Foundation Model" revolution that transformed Natural Language Processing (NLP) and Computer Vision is now reaching time series analysis. As the field shifts from niche, task-specific models to massive Time Series Foundation Models (TSFMs) and LLM-integrated systems (TSLLMs), the primary hurdle has become the "data bottleneck." High-quality time series data is notoriously difficult to aggregate compared to text or images.

The Data Bottleneck

The authors identify four major challenges in collecting real-world time series data:

1. Regulatory Constraints: Privacy laws in healthcare and finance often prevent the sharing of large-scale datasets.
2. Diversity and Bias: Real-world data is frequently imbalanced, over-representing "normal" states while lacking critical anomaly samples.
3. Data Quality: Sensor data is often riddled with noise and missing values.
4. The Multimodal Gap: Emerging tasks like "time series reasoning" require paired numerical data and natural language descriptions, which rarely exist naturally.

Synthetic data offers a way to "program" the training distribution, providing a scalable, unbiased, and private alternative to traditional data collection.

Synthetic Data Generation Strategies

Generation strategies are broadly categorized based on the target model type: numerical TSFMs or multimodal TSLLMs.

1. For Numerical Foundation Models (TSFMs)

These methods focus on generating the "shape" of data using statistical priors:

• Statistical/Kernel-based: Models like Chronos utilize KernelSynth, which randomly combines Gaussian Process (GP) kernels (linear, periodic, RBF) to create diverse patterns.
• Component-based: ForecastPFN and TimesFM decompose series into trend, seasonality, and noise, using linear-exponential functions or ARMA models to simulate realistic dynamics.
• Prior-Data Fitted Networks (PFNs): A novel approach where a model is trained entirely on synthetic data to learn the general properties of "any" time series, enabling zero-shot forecasting on real data.

2. For Large Language Model-based Time Series (TSLLMs)

The challenge here is the alignment between numerical data and natural language. The survey identifies three primary text-generation strategies and various data mixture approaches to bridge this gap.

Expand

Source, p. 5

• Template-based: Uses fixed scripts to describe features (e.g., "The trend is increasing"). While robust, it lacks linguistic variety.
• LLM-based: Employs models like GPT-4 to describe a time series or generate "reasoning" questions about it, providing rich text but risking hallucinations.
• Web-crawled: Aligns financial news or medical reports with corresponding time-stamped data, offering high trustworthiness but suffering from scarcity.

Pretraining and Downstream Tasks

Synthetic data is primarily utilized during the pretraining phase to establish cross-modal alignment and representation learning. By feeding synthetic pairs into TSLLMs, models learn to predict the next token in a sequence, whether that token represents a numerical value or a textual description.

Expand

Source, p. 7

This pretraining enables a wide array of downstream tasks, including forecasting, captioning, reasoning, and multiple-choice question answering (MCQA). Synthetic data has specifically enabled entirely new capabilities, such as Forecast Explanation (FE), by providing the "ground truth" reasoning steps that are absent in raw sensor data.

Key Insights and Results

• The "10% Rule": Research on the Chronos model found that mixing approximately 10% synthetic data with real data yielded the best performance boost. Increasing the synthetic proportion beyond that point often led to decreased performance, suggesting current generators still lack the "messy" complexity of the real world.
• Zero-Shot Supremacy: TSFMs trained on synthetic data (like Mamba4Cast or TimePFN) are now outperforming traditional supervised models (like N-BEATS) on many downstream tasks without seeing a single sample from the target domain.
• The Alignment Paradox: For TSLLMs, the bottleneck is often the "hallucination" in the text. If an LLM generates a description that doesn't perfectly match the numerical trend, the model learns incorrect associations. Consequently, "Easy to Align" synthetic pairs can be more valuable than "Hard to Align" real-world data.

Future Outlook

While synthetic data has proven powerful, several challenges remain. Most synthetic data is still "too clean" and fails to capture the irregular "black swan" events found in real-world systems. Furthermore, there is no rigorous way to ensure LLM-generated descriptions are 100% accurate, leading to potential knowledge contamination.

Future research is expected to focus on adaptive fine-tuning, using synthetic data to target a model's specific weaknesses (like rare anomalies), and self-improvement cycles, where models iteratively generate data, receive human feedback, and refine their own synthetic generators.