DualDynamics: Synergizing Implicit and Explicit Methods for Robust Irregular Time Series Analysis

YongKyung Oh, Dong-Young Lim, Sungil Kim

TL;DR

DualDynamics is a new framework for irregular time series analysis that marries the flexibility of implicit methods (Neural Differential Equations) with the stability and expressive power of explicit methods (Neural Flows). By creating a dual latent space, it achieves state-of-the-art performance in classification, interpolation, and forecasting, even when up to 70% of the data is missing.

What Problem Does This Paper Address?

Real-world time series—like medical records or financial data—are rarely neat. They are often "irregular," meaning observations happen at inconsistent intervals, and "incomplete," with significant gaps in the data.

Currently, researchers use two main approaches, but both have "Achilles' heels":

  1. Implicit Methods (Neural Differential Equations - NDEs): These are great at modeling continuous-time dynamics but struggle with limited expressiveness, high computational costs for long sequences, and numerical stability issues.
  2. Explicit Methods (Neural Flows): These directly map solution curves, making them stable and fast. However, they are notoriously bad at handling irregular sampling and are overly sensitive to the starting point (initial value problem).

The authors identify a "best of both worlds" opportunity: can we use NDEs to handle the messiness of irregular data while using Neural Flows to provide a more expressive and stable representation?

Methods

The authors introduce DualDynamics, which utilizes a two-stage latent representation:

  • The Implicit Component: It uses a Neural Controlled Differential Equation (Neural CDE) to process the raw, irregular input. This component acts as a "bridge," turning discrete, irregular observations into a continuous primary latent state $z(t)$.
  • The Explicit Component: Instead of stopping there, the primary state is fed into a Neural Flow model ($G$). This creates a secondary latent space $\hat{z}(t)$. Because flows are "diffeomorphisms" (invertible and differentiable mappings), they can mold simple distributions into highly complex ones without losing information.
  • Efficient Optimization: To keep the model scalable, the team used adjoint-based backpropagation (calculating gradients without storing all intermediate states) and Hutchinson’s trace estimator to handle the complex math of probability density changes without a massive computational hit.

Datasets

The researchers put DualDynamics through a gauntlet of diverse benchmarks:

  • Classification: 18 datasets from the UEA and UCR repositories (including 'BasicMotions').
  • Interpolation: The PhysioNet Mortality dataset (ICU records).
  • Irregular Sampling: The PhysioNet Sepsis dataset (40,335 patient instances).
  • Forecasting: The MuJoCo physics engine (Hopper configuration) and Google stock data (2011-2021).

Key Insights and Ideas

  • Synergy over Summation: The "Aha!" moment is that the NDE and the Flow aren't just stacked; they evolve synergistically. The NDE handles the "when" and "what" of irregular data, while the Flow handles the "complexity" of the underlying pattern.
  • Stability is King: Standard NDEs often show erratic loss curves during training. DualDynamics demonstrates significantly smoother and more stable convergence, which is critical for deploying these models in the real world.
  • Architecture > Size: The authors proved that the performance gains weren't just because they added more parameters. When compared against a standard MLP with the same parameter count, DualDynamics still won, proving the structure of the dual latent space is what matters.

Key Results

DualDynamics consistently outperformed state-of-the-art models (like Neural CDE, GRU-ODE, and Transformers) across the board:

  • Robustness: In classification tasks with 70% missing data, DualDynamics maintained an accuracy of ~0.697, while traditional RNNs plummeted to ~0.453.
  • Interpolation: On the PhysioNet Mortality task, it achieved a Test MSE of 3.114 (at 90% observation), significantly lower than the next best model, mTAND-Full (4.798).
  • Forecasting: On the MuJoCo dataset, it achieved a minimal MSE of 0.006–0.008, effectively "solving" the prediction task compared to the much higher errors of baseline NDEs.

Limitations and Future Work

While powerful, the authors note a few areas for growth:

  • Flow Selection: Currently, the optimal flow model (ResNet, GRU, or Coupling Flow) is chosen via hyperparameter tuning. Automating this selection or creating a "universal" flow for time series remains an open challenge.
  • More Complex Backends: The framework is modular. The authors suggest that future versions could replace the current flow models with even more sophisticated architectures like Transformers or Temporal Convolutional Networks (TCNs) to further boost expressive power.
  • Computational Overhead: Although more efficient than some NDE variants, the dual-space approach still carries a higher memory footprint than simple RNNs, as seen in the Sepsis dataset memory logs.

Summary updated

Abstract

Real-world time series analysis faces significant challenges when dealing with irregular and incomplete data. While Neural Differential Equation (NDE) based methods have shown promise, they struggle with limited expressiveness, scalability issues, and stability concerns. Conversely, Neural Flows offer stability but falter with irregular data. We introduce 'DualDynamics', a novel framework that synergistically combines NDE-based method and Neural Flow-based method. This approach enhances expressive power while balancing computational demands, addressing critical limitations of existing techniques. We demonstrate DualDynamics' effectiveness across diverse tasks: classification of robustness to dataset shift, irregularly-sampled series analysis, interpolation of missing data, and forecasting with partial observations. Our results show consistent outperformance over state-of-the-art methods, indicating DualDynamics' potential to advance irregular time series analysis significantly.
BibTeX
@inproceedings{oh2025dualdynamics,
  author = {YongKyung Oh and Dong-Young Lim and Sungil Kim},
  title = {{DualDynamics: Synergizing Implicit and Explicit Methods for Robust Irregular Time Series Analysis}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i18.34173},
}
Figure