Introduction
Forecasting correlated time series is a critical task in managing complex cyber-physical systems, such as urban sewage networks or traffic systems. In these environments, sensors record data that is inherently interlinked; for instance, the concentration of one chemical in a treatment plant often influences another due to underlying biological processes. Traditional forecasting methods often struggle with these dynamics due to linearity constraints and a lack of robust feature extraction.
Problem Statement
Standard forecasting approaches face two primary hurdles. First, linear models like ARIMA cannot adequately represent the complex, non-linear interactions within cyber-physical systems. Second, while deep learning models like LSTMs can handle non-linearity, they often process raw, noisy inputs directly. Without specialized feature extraction, these models can overfit to noise or fail to capture the subtle correlations between different series. The objective is to develop a framework that performs automated feature engineering on individual series while simultaneously accounting for their cross-correlations over time.
Proposed Architectures
The authors propose a modular approach combining Convolutional Neural Networks (CNNs) for spatial/local feature extraction and Recurrent Neural Networks (RNNs) for temporal modeling. Two novel architectures are introduced:
1. CRNN (Convolutional Recurrent Neural Network)
In the CRNN model, each individual time series is first fed into its own 1D-CNN. This allows the model to learn "local" patterns and features specific to that variable. These extracted features are then merged into a single vector and passed to an RNN layer to learn sequential dependencies and correlations between the different series.
2. AECRNN (Auto-Encoder CRNN)
The AECRNN builds on the CRNN by adding a deconvolutional (decoder) branch, transforming the model into a multi-task learner.
As illustrated in the AECRNN architecture, the model is trained to minimize a joint loss function consisting of forecasting error and reconstruction error. This auto-encoding task acts as a regularizer, forcing the CNN layers to learn representative, "clean" features of the input by filtering out uncorrelated noise and sensor glitches.
Experimental Evaluation
The models were evaluated on two distinct datasets:
- Sewage Treatment Data: A real-world dataset from Aalborg, Denmark, containing three years of measurements for $NH_4$, $NO_3$, and $O_2$ across six different tanks, sampled every 2 minutes.
- Google Trends: Monthly search popularity for the keywords "potatoes" and "brown sugar" over 14 years, chosen for their high correlation.
Results and Discussion
Experimental results demonstrated that both CRNN and AECRNN consistently outperformed baselines like LSTM, RNN, and ARIMA in terms of Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE). While simple linear models were effective for 1-step-ahead forecasting, their performance deteriorated rapidly as the prediction window ($p$) increased, whereas the proposed deep learning models showed significant strength in long-term forecasting.
Robustness and Interpretability
A key finding was the superior robustness of AECRNN. In experiments with uncorrelated "dummy" noise, AECRNN’s error stayed relatively stable compared to the standard CRNN. Furthermore, visual analysis of the internal "pooling" layers revealed that AECRNN’s filters produced more human-readable representations of the data trends.
As shown in the comparison of forecasting layer inputs, the AECRNN learns smoother transitions and clearer trends, whereas the standard CRNN output appears more chaotic. This suggests that the multi-task objective leads to a more physically meaningful internal state, effectively separating signal from noise.
Conclusion
The CRNN and AECRNN architectures provide a robust framework for forecasting correlated time series. By decoupling feature extraction from temporal modeling and utilizing multi-task learning, these models achieve high accuracy and noise resilience. Future work will investigate the scalability of these architectures to larger sensor networks and their application in real-time transportation network forecasting.
Summary updated