Delay Embeddings for Time Series Representation

21 concepts19 papers1 proposal

When modeling a complex system over time, a single snapshot of the present is rarely enough. Whether predicting the trajectory of a fluid flow, the volatility of a financial market, or the behavior of a power grid, the current state observation $x_t$ is often just a partial projection of a much richer, unobserved dynamical system.

To understand where a system is going, we need to know how it arrived here. The most intuitive way to capture this history is not by building complex, stateful architectures, but by changing how we represent the input data itself. By concatenating or sampling a sequence of historical observations—$[x_t, x_{t-1}, \dots, x_{t-k}]$—we construct a delay embedding (or lagged feature representation). This simple geometric transformation turns a temporal sequence into a static spatial vector, unfolding hidden dimensions of the system and providing models with an explicit sense of memory, momentum, and context.

Reconstructing Phase Space from Lags

Why does stacking past observations work so well? The foundational intuition comes from dynamical systems theory (specifically Takens' Embedding Theorem): if a single observed variable is coupled to other unobserved variables in a system, the history of that single variable contains a trace of the hidden components.

By mapping a low-dimensional time series into a higher-dimensional space of delay coordinates, we can reconstruct the topology of the system's underlying state space. This allows data-driven models to resolve complex temporal dynamics and periodicities that exceed the immediate spatial dimensions of the raw data.

Raw Time Series:  --- x_{t-2} --- x_{t-1} --- x_t ---
                                               |
                                               v
Delay Embedding Vector:              [ x_t, x_{t-1}, x_{t-2} ]

This geometric unfolding is highly effective across diverse domains:

Modeling Hysteresis and Causal Delays

In the real world, cause and effect are rarely instantaneous. Systems exhibit hysteresis—where the current state depends on the path taken to get there—and delayed causal relationships, where an external stimulus takes time to ripple through a system. Lagged features explicitly expose these temporal gaps to a model.

Physical and Mechanical Systems

In physical systems with complex material properties, delay vectors capture mechanical momentum and lag without requiring digital overhead. For instance, in Pose Estimation of a Cable-Driven Serpentine Manipulator Utilizing Intrinsic Dynamics via Physical Reservoir Computing, Masashi Hamaya et al. concatenate past input states into a delay vector. This mapping provides the memory required to account for non-linear behaviors like cable slack and link deformation directly through the robot's physical dynamics. Similarly, in industrial plants, Gang Su et al. show in Transformer-based Drum-level Prediction in a Boiler Plant with Delayed Relations among Multivariates that explicitly pairing a variable with its time-shifted version helps a model navigate the complex, delayed interdependencies inherent to multi-variable industrial processes.

Socio-Economic and Human Systems

Human behavior and logistics also operate on a delay. In DeepExpress: Heterogeneous and Coupled Sequence Modeling for Express Delivery Prediction, Siyuan Ren et al. implement a sliding time window for auxiliary features (like weather or holidays) to account for delayed or anticipatory effects, capturing how external events ripple through delivery volumes over subsequent days.

When modeling human choices, such as transit decisions, Aron Brenner et al. demonstrate in Interpretable Machine Learning Models for Modal Split Prediction in Transportation Systems that incorporating multi-step historical intervals effectively captures the "decision-making window" of users before their choices manifest as aggregate traffic flow data.

Bridging the Gap to Static Machine Learning Models

One of the most practical advantages of lagged feature engineering is that it reframes a sequential time-series problem into a standard supervised learning task. Instead of relying on specialized recurrent architectures (like LSTMs) that maintain an internal hidden state, stacking lags allows simple, static feedforward models to process temporal data.

This strategy dates back to early neural network applications. In A B-P ANN Commodity Trader, Joseph E. Collard demonstrated that expanding input vectors to include multi-day lagged observations provided a basic back-propagation network with the necessary temporal context to recognize market trends and momentum rather than making reactive single-day decisions.

This paradigm remains a staple for forecasting across various domains:

Even in data stream classification, accounting for temporal autocorrelation is vital. Albert Bifet et al. show in Evaluation methods and decision theory for classification of streaming data with temporal dependence that appending the most recent historical labels as input features allows standard classifiers to learn dependencies between consecutive observations without requiring architectural modifications.

Advanced Representations: Signatures and Foundation Models

As machine learning has progressed, the concept of lag features has evolved from simple sliding windows into sophisticated mathematical representations that underpin modern foundation models and unsupervised structural analysis.

The Lead-Lag Embedding in Signature Methods

In rough path theory, sequential data can be mapped into a continuous path and summarized using its "signature" (a collection of iterated integrals). In Embedding and learning with signatures, Adeline Fermanian demonstrates that a "lead-lag" embedding strategy—which augments a time-augmented path with lagged versions of its own coordinates—consistently yields superior performance across diverse sequential datasets. This mathematical formulation ensures that the path's signature explicitly captures the joint variations and temporal relationships between past and present values.

Lag Features as a Foundation Model Primitive

While many modern transformer architectures process time series by breaking them into local "patches" or tokens, this can obscure explicit seasonal frequencies. In Lag-Llama: Towards Foundation Models for Time Series Forecasting, Kashif Rasul et al. build a probabilistic forecasting foundation model that uses sampled lag features as its primary input representation. By sampling historical values at specific time lags that match the data's natural frequency (e.g., lagging by 24 hours for hourly data), the transformer can natively read trends and seasonalities directly from the input features, enabling strong zero-shot generalization on unseen datasets.

Unsupervised Structure and Biological Decoding

Beyond forecasting, delay coordinates allow algorithms to map the macro-structure of a sequence. In Unsupervised Detection of Music Boundaries by Time Series Structure Features, Joan Serrà et al. concatenate windows of recent samples into delay coordinates to create "structure features." This representation accounts for short-term history, enabling the unsupervised segmentation of music tracks into structural blocks like verses and choruses based on homogeneity and repetition.

Similarly, in neuroscience, decoding how the brain represents information requires looking beyond instantaneous states. In Neuronal coding in the rodent prefrontal cortex, Olga Kornienko et al. incorporate time-lagged versions of ensemble activity into their feature sets. This allows decoding models to account for the influence of past states on current neural representations, capturing precise, distributed temporal patterns across cell populations rather than isolated firing rates.

Ultimately, whether applied to the firing of a neuron or the flow of a river, lagging features work because they align with the geometry of time: they transform the hidden history of a system into an explicit, accessible shape.

Key Foundational Work

A B-P ANN Commodity Trader (1990) — contributed 2 concept elements to this direction.
Unsupervised Detection of Music Boundaries by Time Series Structure Features (2012) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_lagged_feature_engineering_7eeadf12,
  title = {lagged feature engineering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lagged-feature-engineering/rdr_7eeadf120bfb41129c4334fd38272b15},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_lagged_feature_engineering_7eeadf12_v1,
  title = {lagged feature engineering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lagged-feature-engineering/rdr_7eeadf120bfb41129c4334fd38272b15/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake