When building machine learning models for dynamic systems—such as evolving knowledge graphs, user transaction histories, or social networks—time is a critical dimension. However, standard approaches that treat time as an absolute coordinate (e.g., specific calendar dates or Unix timestamps) suffer from a fundamental flaw: they cannot easily generalize to the future.
If a model is trained on data from 2020 to 2023, any timestamp from 2024 is inherently out-of-distribution. The model has never seen these numerical values, making direct extrapolation difficult. To overcome this, a powerful paradigm has emerged: relative temporal encoding. Instead of asking "When did this event happen?", models ask "How much time elapsed between these events?" Shifting the frame of reference from absolute coordinates to relative intervals allows models to capture the invariant "tempo" of data, enabling seamless extrapolation to unseen time horizons.
The Intuition: Time-Invariant Tempos
To understand why relative temporal encoding works, consider a sequential recommendation system. If a user purchases a smartphone, they are highly likely to buy a protective case shortly after. The absolute date of the purchase (whether it occurred in 2021 or 2025) is largely irrelevant; what matters is the interval between the two interactions.
As demonstrated by Kei Tateno et al. in Not Just What, But When: Integrating Irregular Intervals to LLM for Sequential Recommendation, modeling these irregular intervals allows systems to capture the true evolution of user preferences. Short intervals indicate immediate, short-term intent, while longer gaps signify long-term preference shifts. By encoding time as a relative difference, the model learns behavioral patterns that remain valid across any chronological era.
Similarly, real-world facts and events possess an underlying rhythm. Pengpeng Shao et al. argue in Adaptive Pseudo-Siamese Policy Network for Temporal Knowledge Prediction that absolute timestamps fail to capture this underlying tempo. By focusing on the duration or delta ($\Delta t$) between consecutive events, models can accurately characterize the frequency and evolutionary patterns of facts, making them far more adept at navigating and predicting future states.
Extrapolation and Lifelong Learning
The primary advantage of relative temporal encoding is its ability to extrapolate to future time steps without retraining. When a model treats time as a transition interval rather than a fixed coordinate, the future looks structurally identical to the past.
Absolute Time (Fails on Future Dates):
[2022-01-01] ---> [2023-01-01] ---> [2024-01-01] (Unseen/OOD)
Relative Time (Generalizes Invariant Patterns):
Event A ---(Δt = 1 year)---> Event B ---(Δt = 1 year)---> Event C
In the context of Temporal Knowledge Graphs (TKGs), forecasting requires predicting events at timestamps completely absent from the training set. Haohai Sun et al. leverage this in TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting by encoding temporal information as a relative offset between a query time and historical event times. Because the model learns patterns based on these offsets, its reinforcement learning agent can "travel" through history and extrapolate to future timestamps with high accuracy, even when encountering entirely new entities.
This concept is extended to streaming data and lifelong learning by Chengjin Xu et al. in Temporal Extrapolation and Knowledge Transfer for Lifelong Temporal Knowledge Graph Reasoning. By treating time as a relative displacement, their framework decouples the reasoning process from specific dates. This allows the model to continuously integrate a stream of new entities and evolving facts over infinite time horizons without suffering from catastrophic forgetting or requiring expensive retraining. As Yuhao Yang et al. note in GraphPro: Graph Pre-training and Prompt Learning for Recommendation, focusing on the duration since an event occurred ensures that the model never encounters out-of-distribution temporal values in production.
Integrating Relative Time into Network Architectures
To use relative time effectively, network architectures must incorporate it directly into their structural operations—specifically within attention mechanisms and neighborhood aggregation.
In graph neural networks, treating time as a structural feature alters how information propagates. For instance, Yuxiao Dong et al. introduce relative temporal encoding to attention mechanisms in the Heterogeneous Graph Transformer. This allows the model to scale to massive, dynamic graphs by maintaining awareness of how relationships fade or strengthen over long periods, capturing shifting node similarities dynamically.
When aggregating information from a node's neighborhood, prioritizing the temporal proximity of interactions yields a more informative representation than chronological ordering alone. Katherine Tieu et al. formalize this in Temporal Graph Neural Tangent Kernel with Graphon-Guaranteed by integrating continuous time-encoding functions directly into the neighborhood aggregation process. Their theoretical framework confirms that modeling the relative duration between interactions provides a more robust guarantee of graph convergence and expressiveness than absolute timestamps.
Furthermore, this relative representation can be paired with path-based reasoning. In Learning to Walk across Time for Interpretable Temporal Knowledge Graph Completion, Jaehun Jung et al. present an attention propagation model (T-GAP) that navigates subgraphs by tracking the "temporal displacement" along edges. This path-based tracking of relative time intervals not only boosts performance on unseen time periods but also provides human-interpretable explanations for predictions based on the historical spacing of events.
Inductive Logic and Rule Transfer
Beyond embedding-based methods, relative temporal representation is vital for learning symbolic, human-readable rules that span across eras. If a logical rule depends on absolute dates, it becomes obsolete as soon as the calendar turns.
To address this, Siheng Xiong et al. developed TILP: Differentiable Learning of Temporal Logical Rules on Knowledge Graphs, a framework that formulates temporal relationships using relative operators (e.g., "before", "after", or specific interval bounds) rather than absolute timestamps. Because the learned rules are based purely on relative temporal intervals, a logical pattern discovered in historical data remains perfectly valid in future or "time-shifted" scenarios, ensuring robust inductive reasoning across entirely different eras.
Key Foundational Work
TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting (2021) — contributed 2 concept elements to this direction.
Learning to Walk across Time for Interpretable Temporal Knowledge Graph Completion (2021) — contributed 1 concept elements to this direction.
Heterogeneous Graph Transformer (2020) — contributed 1 concept elements to this direction.