Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs

Benjamin Walker, Alexandre Bloch, Lingyi Yang, Sam Morley, Terry LyonsarXiv 2026 · arXiv

TL;DR

Continuous-time models like Neural Controlled Differential Equations (NCDEs) typically require interpolating irregular data into continuous paths, which can introduce artificial biases and heavy computational overhead. This paper proves that explicit interpolation is entirely unnecessary if the data embedding is continuous and injective ("faithful"). The authors introduce a novel, interpolation-free embedding for Log-NCDEs that directly summarizes irregular observations into interval log-signatures, achieving up to a 3,000× speedup on long sequences while remaining highly robust to missing data.

What Problem Does This Paper Address?

Real-world time-series data—such as electronic health records or financial logs—is notoriously messy. Observations arrive at irregular intervals, channels are sampled asynchronously, and data is often only partially observed. Standard deep learning architectures (like Transformers or State-Space Models) naturally expect data on a regular grid.

To bridge this gap, continuous-time models embed discrete observations into continuous paths. However, the standard practice relies on imputation or interpolation (e.g., cubic splines) to "fill in the blanks" before the model even sees the data. This design choice forces the model to consume invented, unobserved dynamics and artificial values. The authors of this paper tackle a fundamental question: How can we represent irregular, asynchronous data in continuous time without making arbitrary guesses about unobserved behavior?

Comparison of standard interpolation/imputation approaches versus the proposed faithful representation framework.

Methods

The paper introduces two core contributions to solve this problem:

  1. The Universality Transfer Lemma: The authors mathematically prove that a downstream model's universal approximation capabilities on an input space transfer directly back to the original data space, provided the data embedding is continuous and injective. They term this a faithful embedding. Crucially, the embedding does not need to be smooth.
  2. Direct Interval Log-Signatures: Guided by this theory, they design an embedding specifically for Log-NCDEs. Instead of constructing an interpolated path, they convert discrete observations directly into time-stamped increments and observation counts. These increments are treated as local log-signature contributions within a truncated free Lie algebra.

Using Chen's identity and a parallel associative scan, these local factors are efficiently composed over arbitrary query intervals to form interval log-signatures ($\Phi_{\alpha, \beta}$) without any intermediate interpolation. This approach is integrated into a structure-preserving variant of Structured Linear CDEs, termed Log-SLiCE, enabling massive parallelization.

Schematic showing how interval log-signatures are constructed directly from irregular observations using a parallel associative scan.

Datasets

The authors evaluated their approach across several challenging setups:

  • EigenWorms: A benchmark from the UEA multivariate time-series classification archive featuring exceptionally long sequences (17,984 observations per series).
  • Synthetic Sinusoidal Coupled Data: A custom two-channel system driven by a shared latent phase, evaluated under four distinct sampling regimes (synchronous regular, synchronous irregular, asynchronous irregular, and asynchronous sparse).
  • Brownian Rough Paths: A 4-dimensional Brownian stream on a fine regular grid (2,048 steps) used to drive a Stratonovich linear controlled system, testing the model's capacity to exploit higher-order geometric information (Lévy area).
  • UEA Archive (6 Datasets): A suite of classification datasets used to evaluate model robustness under severe input dropping (up to 95%).

Key Insights and Ideas

  • Stop Guessing, Start Embedding: Traditional preprocessing steps try to guess what happens between observations. This paper argues that guessing should be handled by the modeling layer, not the representation layer. The representation should strictly preserve what was actually observed.
  • Injectivity is All You Need: For a downstream universal model to succeed, the embedding doesn't need to look pretty or smooth; it just needs to keep distinct data objects distinguishable (injective) and ensure nearby data remains nearby (continuous).
  • Observation Counts Matter: By explicitly tracking an "observation count" coordinate, the embedding successfully distinguishes between a channel that repeats its previous value and a channel that simply wasn't measured at that timestamp.
  • Native Higher-Order Information: Because this framework bypasses path construction, it can natively ingest precomputed higher-order geometric features (like Lévy areas), which standard interpolation completely discards.

Key Results

  • Massive Computational Speedups: On the long-sequence EigenWorms dataset, combining the Log-ODE method with a SLiCE architecture (Log-SLiCE) reduced the training time per step by a factor of nearly 3,000× compared to standard NCDEs. This compressed a 3-day training run down to just 1.5 minutes while boosting test accuracy by 11.1 percentage points.
  • Extreme Robustness to Missing Data: Under a severe 95% input dropping regime on the UEA datasets, the block-diagonal Log-SLiCE (BD-SLiCE) maintained an average accuracy of 62.5% (down only slightly from 64.0% with no dropping), significantly outperforming classical recurrent baselines like Mamba and S5.
  • Exploiting Rough Path Geometry: In the Brownian motion experiment, incorporating level-2 log-signatures (area information) yielded substantially lower mean squared error (MSE) than using level-1 increments alone, demonstrating the framework's unique ability to leverage higher-order data structure.
Test MSE against the number of query intervals for the Brownian motion task, demonstrating the advantage of level-2 log-signatures.

Limitations and Future Work

  • Exponential Coordinate Growth: The size of the truncated tensor and log-signature representations scales exponentially with the truncation depth $N$ and the number of channels. Enforcing exact injectivity via auxiliary channels can become computationally expensive for very high-dimensional streams.
  • Invariance Trade-offs: Intentionally omitting count coordinates or physical time introduces mathematical invariances. While this simplifies the model, it can discard critical information in timing-sensitive or event-driven tasks.
  • Empirical Scope: The empirical validation was primarily restricted to classification and synthetic controlled systems. Exploring how this faithful representation performs in large-scale generative modeling or forecasting remains an open direction.

Summary updated

Abstract

Continuous-time models are a natural choice for irregular and asynchronous data. A central design choice is how to embed discrete observations into continuous time. Interpolation- and imputation-based embeddings reconstruct a continuous observation path, making the model sensitive to the choice of reconstruction. We show that this reconstruction step is unnecessary; under mild conditions, compact-set universality on the model input space transfers to the data space whenever the embedding from data to input is continuous and injective. Guided by this result, and building on the rectilinear control path for Neural Controlled Differential Equations (NCDEs), we introduce a continuous and injective embedding for Log-NCDEs, a universal class of continuous-time models. Our approach records observations as increments and composes them over arbitrary query intervals to directly form log-signatures. This provides interval-level summaries without first interpolating the observed variables, while supporting online computation. Experiments on synthetic controlled dynamics and real-world time-series datasets show that the representation is accurate, efficient, and robust to irregular, asynchronous, and sparse observations.
BibTeX
@misc{walker2026faithful,
  author = {Benjamin Walker and Alexandre Bloch and Lingyi Yang and Sam Morley and Terry Lyons},
  title = {{Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs}},
  year = {2026},
  eprint = {2605.30213},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.30213},
}
Figure