Abstract
In the paper, we propose an adaptive data-driven model-based approach for filling the gaps in time series. The approach is based on the automated evolutionary identification of the optimal structure for a composite data-driven model. It allows adapting the model for the effective gap-filling in a specific dataset without the involvement of the data scientist. As a case study, both synthetic and real datasets from different fields (environmental, economic, etc.) are used. The experiments confirm that the proposed approach allows achieving the higher quality of the gap restoration and improve the effectiveness of forecasting models.
Introduction
Time series data—ranging from environmental monitoring to financial market tracking—is critical for predictive modeling. However, this data is rarely continuous; sensor failures, transmission errors, and maintenance periods often result in significant gaps. These missing segments pose a major challenge for high-performance forecasting models that require uninterrupted training data.
While traditional methods like linear interpolation, moving averages, or splines are computationally inexpensive, they often fail to capture the complex, non-linear, and non-stationary patterns inherent in real-world data. More advanced techniques like Kalman filters require expert manual tuning. This paper introduces an automated, adaptive system designed to reconstruct missing data segments without human intervention, ensuring high-quality inputs for downstream tasks.
Methodology: Bi-directional Evolutionary Learning
The proposed approach leverages the FEDOT open-source AutoML framework to evolve "composite models." Instead of relying on a single algorithm, the system constructs complex pipelines or graphs of diverse machine learning blocks (e.g., Lasso regression, K-Nearest Neighbors).
The core innovation is a bi-directional forecasting strategy. Rather than predicting forward from the pre-gap history alone, the algorithm treats the gap as a target for two converging forecasts:
- Forward Prediction (A): Uses the data preceding the gap to forecast the missing values.
- Backward Prediction (B): Inverts the post-gap data to forecast "backward" into the gap.
These two predictions are combined into an ensemble. The optimal structure of this ensemble and its constituent models is determined through an evolutionary process involving crossover and mutation operators, which optimize a fitness function (such as RMSE) over multiple generations.
Experimental Evaluation
The researchers evaluated the robustness of the approach across five distinct types of time series:
- Synthetic data: Artificially generated series with known mathematical properties.
- Sea level (Hourly & Daily): High and low-frequency environmental data.
- Air temperature: Standard meteorological observations.
- Economic time series: Highly volatile financial data.
Experimental gaps were substantial, sometimes accounting for up to 30% of the total series length, including long-duration gaps of up to 1,500 steps.
Results and Discussion
The proposed evolutionary approach significantly outperformed traditional baselines across all tested scenarios. The adaptability of the AutoML framework allowed it to tailor model architectures to the specific characteristics of each dataset—such as the seasonality of temperature data or the volatility of economic trends.
As shown in the results table, the improvement was most pronounced in the economic dataset, where the proposed approach achieved a MAPE of 32.4, compared to 193.1 for linear interpolation. Across all datasets, the proposed solution achieved an average MAPE of 15.7%. Crucially, when the restored series were used to train subsequent forecasting models, the "Proposed approach" led to the lowest forecast error (11.2 MAPE). This demonstrates that the reconstructed data is not just visually plausible but maintains the underlying statistical integrity required for further machine learning applications.
Conclusion
This research demonstrates that an automated, evolutionary approach to gap filling can provide superior restoration quality compared to traditional interpolation methods. By formalizing bi-directional forecasting within an AutoML framework, the system removes the need for manual model selection and tuning. Future work will focus on extending this methodology to multivariate time series, where correlations between different variables can be exploited to further improve restoration accuracy.
Summary updated