MODE: Efficient Time Series Prediction with Mamba Enhanced by Low-Rank Neural ODEs

Xingsheng Chen, Regina Zhang, Bo Gao, Xingwei He, Xiaofeng Liu, Pietro Lio, Kwok-Yan Lam, Siu-Ming YiuarXiv 2026 · arXiv · OpenReview

TL;DR

This paper introduces MODE, a novel framework for time series prediction that combines the Mamba architecture with Low-Rank Neural Ordinary Differential Equations (Neural ODEs). MODE achieves state-of-the-art accuracy and efficiency by effectively handling long-range dependencies and irregularly sampled data, outperforming existing methods on various benchmarks.

What Problem Does This Paper Address?

Time series prediction is crucial across many fields, but existing methods often struggle to balance accuracy, efficiency, and scalability, especially when dealing with long-range dependencies and irregularly sampled data. Traditional approaches like RNNs and Transformers can be computationally expensive and have difficulty capturing subtle temporal dynamics. Irregularly sampled data, common in healthcare or sensor networks, further complicates prediction by violating the uniform time-step assumption of many models. This paper aims to bridge this gap by proposing a unified framework that addresses these challenges.

Methods

MODE integrates two key innovations:

  • Low-Rank Neural ODEs: Instead of relying on discrete time-step updates, MODE uses Neural ODEs to model state transitions as continuous-time dynamics. This allows for natural handling of irregularly sampled data. To manage computational complexity, the state transition matrix is approximated using a low-rank decomposition, reducing its complexity from O(d²) to O(d·r), where 'r' is the rank and 'd' is the hidden dimension.
  • Enhanced Mamba Architecture with Selective Scanning: MODE builds upon the Mamba architecture, known for its efficiency in sequence modeling. It incorporates a "segmented selective scanning" mechanism. This mechanism adaptively focuses on the most informative segments of the time series, further improving computational efficiency and scalability for long sequences.

The framework first embeds the input time series, then generates low-rank matrices for the ODEs, performs dynamic selective scanning within segments, and finally concatenates segment outputs for prediction.

Datasets

The paper evaluates MODE on a diverse set of 8 public real-world datasets:

  • ETTh1, ETTh2, ETTm1, ETTm2 (likely related to energy or load forecasting)
  • Electricity
  • Exchange
  • Weather
  • Solar Energy
  • PEMS08 (likely related to traffic data)

Key Insights and Ideas

  • Continuous-time dynamics are key for irregular data: Using Neural ODEs allows the model to naturally handle data points that are not evenly spaced in time, a significant advantage over discrete-time models.
  • Low-rank approximation is a powerful efficiency lever: Reducing the complexity of the state transition matrix through low-rank decomposition significantly cuts down on computation and memory, making the model scalable without sacrificing too much representational power.
  • Selective scanning enhances focus: By intelligently selecting which parts of the sequence to process, the model avoids redundant computations and focuses its resources on the most relevant temporal information.
  • Synergy between Mamba and Neural ODEs: The combination of Mamba's efficient sequence processing with the continuous-time modeling of Neural ODEs creates a powerful and robust framework.

Key Results

  • State-of-the-art accuracy: MODE consistently outperforms strong baselines (including S-Mamba, iTransformer, and PatchTST) across various datasets and prediction horizons, achieving the lowest MSE and MAE on average.
  • High efficiency: MODE demonstrates significantly lower training time and memory usage compared to Transformer-based and other Mamba variants, making it practical for large-scale applications. For instance, it uses only 0.037 GB of memory on ETTm1.
  • Robustness: The model maintains stable performance even with moderate levels of Gaussian noise injected into the input sequences, showing resilience to noisy environments.
  • Long-term forecasting capability: MODE consistently achieves low errors even with very long lookback windows (up to 720 time steps), outperforming methods that struggle with long-range dependencies.

Limitations and Future Work

  • Limitations: The paper doesn't explicitly detail limitations in the provided text, but the inherent trade-off between rank size and model capacity is a common consideration in low-rank approximations.
  • Future Work: The authors suggest future directions such as integrating probabilistic inference and physics-informed priors to further enhance uncertainty quantification and broaden the model's applicability to real-world scenarios.

Summary updated

Abstract

Time series prediction plays a pivotal role across diverse domains such as finance, healthcare, energy systems, and environmental modeling. However, existing approaches often struggle to balance efficiency, scalability, and accuracy, particularly when handling long-range dependencies and irregularly sampled data. To address these challenges, we propose MODE, a unified framework that integrates Low-Rank Neural Ordinary Differential Equations (Neural ODEs) with an Enhanced Mamba architecture. As illustrated in our framework, the input sequence is first transformed by a Linear Tokenization Layer and then processed through multiple Mamba Encoder blocks, each equipped with an Enhanced Mamba Layer that employs Causal Convolution, SiLU activation, and a Low-Rank Neural ODE enhancement to efficiently capture temporal dynamics. This low-rank formulation reduces computational overhead while maintaining expressive power. Furthermore, a segmented selective scanning mechanism, inspired by pseudo-ODE dynamics, adaptively focuses on salient subsequences to improve scalability and long-range sequence modeling. Extensive experiments on benchmark datasets demonstrate that MODE surpasses existing baselines in both predictive accuracy and computational efficiency. Overall, our contributions include: (1) a unified and efficient architecture for long-term time series modeling, (2) integration of Mamba's selective scanning with low-rank Neural ODEs for enhanced temporal representation, and (3) substantial improvements in efficiency and scalability enabled by low-rank approximation and dynamic selective scanning.
BibTeX
@misc{chen2026mode,
  author = {Xingsheng Chen and Regina Zhang and Bo Gao and Xingwei He and Xiaofeng Liu and Pietro Lio and Kwok-Yan Lam and Siu-Ming Yiu},
  title = {{MODE: Efficient Time Series Prediction with Mamba Enhanced by Low-Rank Neural ODEs}},
  year = {2026},
  eprint = {2601.00920},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.00920},
}
Figure