Periodic Temporal Embeddings for Time Series Modeling

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2396 papers502 conceptsfrozen 2026-10-04

Standard neural networks often treat time as a simple linear counter or a raw scalar timestamp. However, time in the real world is rarely just a straight line; it is a complex tapestry of repeating cycles—hours, days, weeks, and seasons—overlaid on top of long-term trends. If a model only sees time as a growing number, it fails to recognize that 11:59 PM and 12:01 AM are chronologically adjacent, or that a Monday morning in July shares structural similarities with a Monday morning in August.

To bridge this gap, researchers have developed Periodic Temporal Embeddings. This family of techniques transforms raw time into high-dimensional vectors that explicitly capture the cyclical and multi-scale nature of temporal data.

The Intuition: Mapping Time to a Circle

The fundamental challenge of temporal data is the "discontinuity" of linear representations. In a 24-hour clock, the numerical jump from 23 to 0 is large, even though the actual time difference is minimal.

As demonstrated by Yuji Horiguchi and Yukino Baba in their work on flight delay prediction, the solution is to map time points onto a circular space using trigonometric functions like sine and cosine. By representing a timestamp as a coordinate on a unit circle, the model naturally perceives the proximity between the end of one cycle and the beginning of the next. This ensures that the transition from December 31st to January 1st is smooth rather than a cliff-edge jump in the feature space.

While cycles are vital, time also has a non-periodic component: the world changes over years and decades. A robust representation must capture both the "circle" (periodicity) and the "line" (progression).

Seyed Mehran Kazemi and Rishab Goel formalized this in Time2Vec, a model-agnostic vector representation of time. Time2Vec decomposes time into two parts:

  1. A linear term that tracks the steady progression of time, allowing the model to handle long-term trends.
  2. A set of periodic terms (sine-based transformations) that capture recurring patterns.

This hybrid approach allows a model to understand that while today is "like" yesterday in terms of the daily cycle, it is also distinct because it occurs further along a linear trajectory. Similar logic is applied in TCAN by Xigang Sun et al., which uses explicit embeddings to model the power-law growth and rhythmic decay of information virality.

Multi-Scale and Feature-Dependent Rhythms

Not all cycles are created equal. A mobility model needs to distinguish between a daily commute and a weekend outing, while a clinical model might need to track the specific 8-hour cycle of a medication dose.

  • Multi-Scale Encoding: In the Pretrained Mobility Transformer, Haoyu He and colleagues use multi-scale embeddings to integrate absolute sequence positions with periodic signals for days and weeks. This allows the model to "read" the rhythm of a city.
  • Feature-Dependent Rhythms: Different events have different frequencies. The XTSFormer architecture introduces feature-dependent periodic encoding, where the model learns specific frequencies for different types of events. For example, it might learn a high-frequency encoding for a patient's heart rate but a lower-frequency one for their weekly check-ups.
  • Personalized Rhythms: In temporal graphs, Binbin Hu and Jun Zhou proposed MERIT, which learns Fourier-series coefficients for each specific node. This allows the model to adapt to individual-level habits, such as a specific user's unique monthly shopping routine.

Beyond Forecasting: Quantization and Physics

The utility of periodic embeddings extends beyond simple time-series forecasting. In the realm of generative AI, Junhyuk So and Eunhyeok Park developed Temporal Dynamic Quantization for diffusion models. By applying geometric Fourier encoding to the scalar denoising time-step, they enable the model to capture high-frequency shifts in internal activations, allowing for efficient 4-bit generation without losing quality.

Similarly, in physics-informed modeling, Marlon Sproesser Mathias and Fabio Cozman use trigonometric basis functions to model tidal flows in port channels. By baking periodicity directly into the architecture, the model converges to steady-state solutions much faster than a standard network would, effectively "respecting" the physical laws of the tide.

Summary

Periodic temporal embeddings move neural networks away from treating time as an arbitrary index and toward treating it as a structured, multi-dimensional signal. By leveraging trigonometric functions to handle circular continuity and combining them with linear terms to track progress, these representations allow models to capture the inherent rhythms of the physical and social world. As Qingsong Wen notes in a comprehensive survey, these techniques are often the deciding factor in whether a Transformer can successfully generalize to long-sequence, real-world time series data.

Papers in this version

Showing 60 of 396 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 240 papers
  2. Version 1 · 2026-10-04 · 396 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_cyclical_temporal_encoding_f830de0a,
  title = {cyclical temporal encoding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cyclical-temporal-encoding/rdr_f830de0a02c14eb9ab29b15bdaa2e561},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_periodic_temporal_embeddings_for_time_se_f830de0a_v1,
  title = {Periodic Temporal Embeddings for Time Series Modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/periodic-temporal-embeddings-for-time-series-modeling/rdr_f830de0a02c14eb9ab29b15bdaa2e561/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}