Optimal starting point for time series forecasting

Yiming Zhong, Yinuo Ren, Guangyao Cao, Feng Li, Haobo Qi

TL;DR

This paper introduces OSP-TSP (Optimal Starting Point Time Series Forecast), a framework that improves forecasting accuracy by identifying the best historical point to begin training. By using machine learning to detect structural breaks and concept drifts, the method automatically truncates irrelevant past data, consistently outperforming models trained on entire datasets.

What Problem Does This Paper Address?

In the world of big data, the prevailing wisdom is often "the more data, the better." However, time series data is rarely stationary. Real-world events—like the COVID-19 pandemic, policy shifts, or sudden market crashes—create structural breaks or concept drifts.

If you are trying to predict tourism trends in 2024, including data from the 1980s might actually hurt your model because the underlying "rules" of travel have changed. While some models (like LSTM or ETS) naturally give more weight to recent observations, they still "see" the old patterns, which can introduce noise. The authors tackle the challenge of automatically determining exactly where a time series should "start" to minimize future prediction errors, moving away from subjective or manual data truncation.

Methods

The authors propose a two-step framework called OSP-TSP:

  1. The Meta-Learning Step (Finding the OSP): Instead of guessing, the authors train a supervised machine learning model (XGBoost or LightGBM) to predict the optimal starting interval.

  2. Feature Extraction: They use the tsfeatures R package to turn raw time series into a vector of descriptive statistics (e.g., trend strength, seasonality, linearity, curvature, and autocorrelation).

  3. Interval Prediction: To make the problem computationally feasible, they don't look for a specific point but rather divide the series into $m$ sub-intervals. The meta-model learns which interval contains the starting point that results in the lowest forecast error.
  4. The Forecasting Step: Once the OSP is identified, the series is truncated. The authors then apply standard "base" forecasting models—such as ETS (Exponential Smoothing), thetaf, ARIMA, and nnetar (Neural Network Autoregression)—to the shortened series to produce the final forecast.

Datasets

The study relies on a massive and diverse set of data:

  • M4 Dataset: A benchmark in the forecasting community containing 100,000 time series across various frequencies (yearly, quarterly, monthly, weekly, daily, hourly) and domains (finance, macro, micro, etc.).
  • Real-World Economic Indicators: Five specific datasets including GDP, Construction Industry stats, Exchange Rates, Confidence Indices, and Import Values.
  • Simulated Data: They also use the GRATIS method to generate synthetic time series to test if a model pre-trained on simulated data can generalize to real-world tasks.

Key Insights and Ideas

  • The "Less is More" Paradox: The most significant finding is that using a subset of recent data consistently yields better results than using the full historical record, especially when the series has undergone a shift.
  • Feature Importance: The authors discovered that Curvature and Linearity are the most pivotal features for determining the optimal starting point across almost all seasonal models. For high-frequency data (weekly/daily), autocorrelation features after differencing become the decisive factors.
  • Transferability: You don't necessarily need a massive dataset for every new task. An OSP model pre-trained on the M4 dataset or even on simulated data can effectively find the optimal starting points for entirely different real-world datasets.
  • Interval Averaging: Instead of picking one "perfect" start point, the authors found that picking several points within the predicted "optimal interval" and averaging their forecasts provided a more stable and accurate result.

Key Results

  • Consistency: Across yearly, quarterly, and monthly data in the M4 set, the OSP-TSP approach reduced the MASE (Mean Absolute Scaled Error) compared to using the total series. For example, in yearly data using an ETS base model, the MASE dropped from 3.44 (total series) to as low as 2.51 (OSP-TSP).
  • Base Model Agnostic: The method improved performance regardless of whether the base model was a simple statistical one (ARIMA) or a complex neural network (nnetar).
  • Superior to Change-Point Detection: The authors compared OSP-TSP against standard CUSUM-based change-point detection. OSP-TSP performed better because it specifically optimizes for forecast error rather than just looking for statistical shifts in the mean or variance.

Limitations and Future Work

  • Computational Cost: Training the meta-model to find the OSP for 100,000 series is "laborious" and computationally demanding. While they mitigated this by using sub-intervals, it remains a hurdle for real-time applications.
  • Metric Sensitivity: The OSP was optimized for MASE. While this improved MASE significantly, the improvements in MAPE (Mean Absolute Percentage Error) were sometimes less consistent, suggesting the choice of target metric in the meta-learning phase is critical.
  • Selection Criteria: The paper notes that the criteria for selecting the "best" interval (minimum error vs. average error) can impact results, and more work is needed to refine these selection heuristics.
  • Future Directions: The authors suggest exploring parallel computing to speed up the process and investigating more diverse base models to see if the OSP benefits scale with even more complex architectures.

Summary updated

BibTeX
@article{zhong2025optimal,
  author = {Yiming Zhong and Yinuo Ren and Guangyao Cao and Feng Li and Haobo Qi},
  title = {{Optimal starting point for time series forecasting}},
  journal = {Expert Systems with Applications},
  year = {2025},
  doi = {10.1016/j.eswa.2025.126798},
}
Figure