Apollo-Forecast: Overcoming Aliasing and Inference Speed Challenges in Language Models for Time Series Forecasting

Tianyi Yin, Jingwei Wang, Yunlong Ma, Han Wang, Chenze Wang, Yukai Zhao, Min Liu, Weiming ShenAAAI 2025 · DOI

TL;DR

This paper introduces Apollo-Forecast, a novel framework that tackles two major challenges in using large language models (LLMs) for time series forecasting: aliasing distortion during tokenization and slow inference speeds. By introducing an Anti-Aliasing Quantization Module (AAQM) and a Race Decoding (RD) technique, Apollo-Forecast significantly improves forecasting accuracy and dramatically accelerates inference, making LLMs more practical for real-world time series prediction.

What Problem Does This Paper Address?

Time series forecasting is crucial across many domains, from finance to energy. While traditional statistical and machine learning models have been effective, they often struggle with complex patterns and long-term dependencies. The advent of large language models (LLMs) has opened new avenues for time series forecasting due to their impressive generalization capabilities. However, applying LLMs to time series data presents unique hurdles. One significant issue is aliasing distortion, which occurs when encoding time series data into discrete tokens. This process, especially with fixed token lengths, can cause high-frequency noise to fold into the low-frequency band, corrupting the signal and reducing forecasting accuracy. Another major challenge is the prolonged inference time of LLMs, particularly for long-term predictions. This slowness, stemming from the autoregressive nature of LLMs and their large model sizes, limits their practical applicability in real-time scenarios. This paper aims to bridge this gap by developing a method that enhances both the accuracy and efficiency of LLM-based time series forecasting.

Methods

Apollo-Forecast introduces two core innovations:

  • Anti-Aliasing Quantization Module (AAQM): This module addresses the aliasing distortion problem. It comprises two key components:
  • Noise Erasure (NE): AAQM employs a Butterworth filter to remove high-frequency noise from the original time series signal before quantization. This preserves the low-frequency components, which are crucial for forecasting, and significantly improves the signal-to-noise ratio. The paper mathematically demonstrates how this process reduces aliasing errors without affecting the signal's phase.
  • Time Series Embedding Module: After noise erasure, the signal is normalized and quantized. The quantized values are then mapped to a discrete set of tokens using an embedding function, preparing them for the LLM.
  • Race Decoding (RD): This technique is designed to accelerate inference speed without altering the LLM's structure. RD utilizes a two-model approach:
  • Draft Model (Fast): A smaller, faster model is used to make initial predictions.
  • Main Model (Slow): The primary, larger LLM performs inference. Both models predict in parallel. After the draft model completes a portion of the prediction horizon, a Tolerance Check is performed. If the draft model's predictions are sufficiently close to the main model's (within a defined error threshold $\gamma$), the results are concatenated. This allows for faster completion by leveraging the draft model's speed for a part of the prediction. If the tolerance check fails, the system waits for the main model to complete its inference, ensuring accuracy is maintained. The theoretical inference time for RD is thus $\min(T_{draft} + T_{tolerance}, T_{main})$.

Datasets

The paper evaluates Apollo-Forecast on several real-world datasets:

  • UCR Dataset: A collection of 114 time series records from diverse domains like health, energy, and traffic. This dataset is used to evaluate the model's generalization capabilities under zero-shot conditions.
  • Public Datasets: Seven types of records were used: ETTh1, ETTh2, ETTm1, ETTm2, Weather, ECL, and TrafficL. These are standard benchmarks for time series forecasting.
  • LBS Dataset: This dataset consists of real-time pedestrian flow data from Shanghai, recorded at 5-minute intervals. It was used to assess the model's performance in a specific application scenario.

Key Insights and Ideas

  • Aliasing is a fundamental problem in time series tokenization: The paper clearly illustrates how standard quantization methods can introduce significant distortion by folding high-frequency noise into the low-frequency band, directly impacting forecasting accuracy.
  • Pre-filtering noise is more effective than post-hoc correction: By proactively removing high-frequency noise before quantization using AAQM, the model preserves crucial low-frequency information, leading to better signal fidelity and improved forecasting performance.
  • Leveraging a "draft" model for speed-up is viable: The Race Decoding technique demonstrates that a hybrid approach, combining a fast draft model with a more accurate main model, can significantly accelerate inference without sacrificing accuracy, especially for long horizons.
  • AAQM has a more substantial impact on accuracy than RD on speed: Ablation studies show that removing AAQM drastically worsens performance, while disabling RD primarily affects speed with only marginal impacts on accuracy.

Key Results

  • Accuracy Improvement: Apollo-Forecast achieves significant improvements in zero-shot scenarios. On the UCR dataset, it shows reductions of up to 35.41% in Agg. Relative WQL and 17.01% in Agg. Relative MASE compared to state-of-the-art methods. On public datasets, it achieves reductions of up to 31.44% in WQL and 18.99% in MASE.
  • Inference Speedup: The Race Decoding technique provides substantial speed improvements. On the UCR dataset, it achieves an acceleration of 1.9X-2.7X in inference speed.
  • Robustness: The model demonstrates strong performance across various datasets and prediction horizons, highlighting its generalization capabilities.

Limitations and Future Work

  • Limitations:
    • The paper notes that while RD accelerates inference, it relies on the assumption that the draft and main predictors can approximate the same probability distribution. Model randomness and distribution drift can lead to differences, necessitating the tolerance check.
  • The effectiveness of RD is more pronounced for longer prediction horizons.
  • Future Work:
    • The authors plan to refine the framework further.
  • They aim to explore Apollo-Forecast's applicability to other domains beyond forecasting, such as finance, weather forecasting, and manufacturing.
  • Future work will focus on enhancing the generalization capability and efficiency of Apollo-Forecast in multi-frequency scenarios.

Summary updated

BibTeX
@inproceedings{yin2025apollo,
  author = {Tianyi Yin and Jingwei Wang and Yunlong Ma and Han Wang and Chenze Wang and Yukai Zhao and Min Liu and Weiming Shen},
  title = {{Apollo-Forecast: Overcoming Aliasing and Inference Speed Challenges in Language Models for Time Series Forecasting}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i21.34371},
}
Figure