TL;DR
SLiDE (Structured Linear Dynamics with Exogenous inputs) is a lightweight, Koopman-inspired forecasting model designed to handle external drivers like weather or promotions. By enforcing structured linear dynamics in a latent space, it avoids overfitting to historical noise and significantly outperforms state-of-the-art models when external factors shift outside their training range.
What Problem Does This Paper Address?
In many real-world scenarios—like predicting electricity prices or retail demand—the target variable is heavily influenced by "exogenous inputs" (external factors like wind power forecasts or holiday schedules). While modern deep learning models are excellent at finding patterns, they often overfit to specific historical correlations.
When these external factors shift—for example, if a wind farm produces more power than ever recorded during training—standard models often fail to generalize. The authors identify a need for models that don't just fuse features together but actually learn the underlying "input-response" behavior of the system to remain robust under distribution shifts.
Methods
SLiDE draws inspiration from Koopman theory, which posits that nonlinear dynamical systems can be represented as linear systems if they are "lifted" into a sufficiently high-dimensional latent space.
Key architectural components include:
- Learned Linear Operator ($W$): Instead of complex nonlinear transitions, SLiDE uses a learned matrix to evolve the latent state.
- Structured History Encoder: A lag-invariant recurrent encoder that reconstructs the current latent state from past targets and inputs. Crucially, it uses the same linear recurrence structure as the prediction phase.
- Shared Linear Rollout: The model predicts the future by rolling the latent state forward using the same matrix $W$, driven by additive "forcing" from future exogenous variables.
- Block Structure: The latent state is decomposed into $N$ independent subspaces (blocks), allowing the model to capture different dynamical modes (e.g., fast vs. slow changes) in parallel.
- Nonlinear Output Map: While the internal dynamics are linear, the final projection back to the target space uses a small MLP to handle remaining approximation errors.
Datasets
The authors evaluated SLiDE on a diverse mix of real-world and synthetic benchmarks:
- EPF (Electricity Price Forecasting): Five datasets from different electricity markets (NP, PJM, EPEX BE, FR, and DE).
- Favorita: A large-scale retail sales dataset involving over 1,500 product series with promotion and holiday data.
- Bouc-Wen System: A synthetic nonlinear hysteretic system used to test "transfer" capabilities (training on one type of input process and testing on another).
- Additional Benchmarks: UCI Bike Sharing and a synthetic Duffing oscillator were used for further validation against system-identification baselines.
Key Insights and Ideas
- Consistency as a Constraint: By using the exact same linear operator to encode the past and predict the future, the model is forced to learn a mathematically consistent representation of the system's physics.
- Graceful Extrapolation: Linear dynamics provide a powerful inductive bias for "out-of-range" (OOR) scenarios. Unlike deep nonlinear networks that might produce erratic outputs when seeing unprecedented input values, a linear rollout extrapolates more predictably.
- Efficiency via Structure: Despite its performance, SLiDE maintains a computational footprint similar to lightweight MLP architectures (like TSMixer) and is significantly faster than Transformer-based models like TimeXer.
Key Results
- State-of-the-Art Accuracy: SLiDE achieved the best average rank across all benchmarks compared to competitive forecasting architectures and Koopman-based baselines.
- Robustness to Shifts: On "out-of-range" electricity price samples, SLiDE reduced Mean Squared Error (MSE) by 20% compared to the next best model.
- Synthetic Transfer: In the Bouc-Wen transfer task, it reduced Mean Absolute Error (MAE) by 30%, proving it can generalize to entirely unseen input processes.
- Ablation Success: Removing either the structured encoder or the linear rollout led to significant performance drops (e.g., an 18% increase in error on OOR Favorita samples), confirming that both components are vital for robustness.
Limitations and Future Work
- Linearity Assumption: SLiDE's effectiveness depends on the forecasting problem being reasonably approximable by linear latent dynamics. Systems with extremely rich or continuous spectra might be too restrictive for this architecture.
- Input Sensitivity: Because exogenous inputs directly drive the latent rollout, the model's accuracy is tied to the quality of future exogenous forecasts.
- Future Directions: The authors suggest exploring more adaptive latent operators (context-dependent variants) and extending the analysis to a wider range of covariate shifts beyond just "out-of-range" values.
Summary updated