Explainable Parallel RCNN with Novel Feature Representation for Time Series Forecasting

Jimeng Shi, Rukmangadh Myana, Vitalii Stebliankin, Azam Shirali, Giri Narasimhan

Introduction

Time series forecasting is a cornerstone of modern data science, powering everything from retail demand planning to managing hospital bed capacity. However, most models struggle when external factors—like a sudden storm or a scheduled dam gate opening—influence the outcome. A new paper by Shi et al. proposes a clever architectural shift to handle these "future covariates" more effectively through a framework called ParaRCNN.

The Challenge of Future Covariates

Most time series forecasting falls into two camps: iterative (predicting one step, then using that prediction to find the next) and direct (predicting the whole window at once, often using Seq2Seq/Encoder-Decoder architectures).

Iterative models suffer from "error accumulation," where small mistakes in early steps snowball. Direct Seq2Seq models are more robust but typically process the past (encoder) and the future (decoder) separately. The authors argue this separation prevents models from fully capturing the interactions between historical trends and future drivers. For example, to predict a flood, a model needs to contextually link a future rain forecast (future covariate) with current river levels (past data) simultaneously.

Proposed Methodology

The paper introduces two major innovations to bridge the gap between historical data and future predictions:

1. Data Fusion via Shifting

Instead of relying on a traditional encoder-decoder split, the authors "shift" future predictable covariates back in time. This aligns future information (e.g., tomorrow's weather) with historical data (e.g., today's temperature) into a single unified input. This allows the model to "see" the relationship between past and future at every time step within a single, compact representation.

The shifting strategy aligns future predictable covariates with historical data to create a unified input representation.

2. ParaRCNN Architecture

To process this fused data, the authors developed a parallel framework designed to extract diverse temporal features:

  • RNN Branch: Specifically designed to capture long-term temporal dependencies and the "flow" of time.
  • CNN Branch: Uses 1D convolutional filters to extract local patterns and cross-feature dependencies (the "shape" of local events).
  • Hierarchical Structure & Skip Connections: Both branches use a hierarchical structure to distill high-level features. They utilize concatenation-based skip connections that pass the original input to every subsequent layer, maximizing feature reuse and preventing vanishing gradients.
The ParaRCNN architecture features parallel CNN and RNN branches with dense skip connections to process fused inputs.

Experimental Results

The model was evaluated on three diverse real-world datasets: Beijing PM2.5 (air quality), Electricity Price (market demand), and Water Stage (river levels). ParaRCNN consistently outperformed established baselines like DeepAR, MQRNN, and the Temporal Fusion Transformer (TFT) across all metrics.

  • Water Stage Dataset: ParaRCNN achieved an RMSE of 0.075, significantly lower than TFT's 0.158.
  • Beijing PM2.5: It reached an RMSE of 49.97, compared to MQRNN’s 53.13.

Key Insights

  • The "Shifting" Sweet Spot: Performance is optimized when the shift length is carefully tuned between the forecasting horizon and the look-back window.
  • Parallelism is Power: By running RNNs and CNNs in parallel, the model captures both global trends and local anomalies more effectively than sequential or single-branch models.
  • Interpretability: Using an adapted Grad-CAM approach, the authors demonstrated that the model correctly prioritizes future covariates when making predictions, validating the logic of the shifting strategy.

Conclusion

ParaRCNN offers a more compact and efficient framework for time series forecasting by unifying past and future data. While it relies on the availability of reasonably accurate future covariates, its ability to outperform complex transformer-based models like TFT suggests that architectural simplicity and clever feature representation can be more effective than sheer model depth.

Summary updated

BibTeX
@inproceedings{shi2023explainable,
  author = {Jimeng Shi and Rukmangadh Myana and Vitalii Stebliankin and Azam Shirali and Giri Narasimhan},
  title = {{Explainable Parallel RCNN with Novel Feature Representation for Time Series Forecasting}},
  booktitle = {AALTD@ECML/PKDD},
  year = {2023},
  doi = {10.1007/978-3-031-49896-1\_5},
}
Figure