In many real-world reasoning tasks, information is not a uniform stream. Instead, it is a mixture of two fundamentally different types of data: permanent facts (e.g., a person’s place of birth) and evolving events (e.g., a person’s current location).
Standard machine learning models often struggle with this heterogeneity. If a model treats all data as a flat sequence, the "noise" of constant, time-invariant facts can overwhelm the subtle signals of temporal change. Conversely, if a model focuses only on recent dynamics, it loses the grounding provided by static background knowledge. Recent research across dialogue systems and Temporal Knowledge Graphs (TKGs) has converged on a shared solution: dual-path architectures that explicitly separate and then selectively fuse static and dynamic information.
The Structural Mismatch
The core problem stems from a structural mismatch between different types of information. In dialogue systems, Kai Hua and Rui Yan observe in Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems that conversation turns are temporally dependent—the meaning of "Yes" depends entirely on the previous question. However, external knowledge facts (like a user's persona) are often independent sets of information.
When models use a single architectural approach—like a standard Transformer or RNN—for both types of data, they ignore these distinct relational characteristics. Zehao Lin and Yin Zhang addressed this in Task-Oriented Conversation Generation Using Heterogeneous Memory Networks by proposing Heterogeneous Memory Networks (HMNs). Their approach splits the model's memory into two:
- Context-aware memory: Uses sequential modeling to capture the flow of dialogue.
- Context-free memory: Uses set-based attention to store and retrieve static facts.
Static Facts as Search Filters
In the realm of Temporal Knowledge Graphs, the challenge is even more acute. A TKG might contain millions of facts, some valid for a day and others valid forever. Zifeng Ding and Yan Xia argue in Temporal Fact Reasoning over Hyper-Relational Knowledge Graphs that static facts should not just be "added" to the model; they should act as a filter.
If a model knows a person’s native language is French (a static fact), it can significantly narrow the search space when trying to predict which city they might visit next (a temporal event). By using static attributes to constrain entity representations, the model avoids wasting "computational energy" on impossible predictions.
The Gated Integration Strategy
The most difficult part of this research is the fusion step: how do you combine these two paths without the static data drowning out the dynamic data?
In Temporal Fact Reasoning over Hyper-Relational Knowledge Graphs, Bo Xiong and colleagues identify a phenomenon where models experience performance degradation when static facts are introduced naively. To solve this, they utilize a gated integration structure. This gate acts like a traffic controller, deciding for each entity and relation how much the permanent background should influence the current temporal prediction. This prevents the "permanent" signal from overwhelming the "evolving" signal, ensuring that the model remains sensitive to change.
Multi-Dimensional and Tensor Perspectives
Other researchers have looked at this duality through the lens of mathematical geometry. Pengpeng Shao and Jianhua Tao developed TuckERT (Tucker decomposition-based Temporal Knowledge Graph Completion), which treats the problem as a 4th-order tensor decomposition. By adding a dedicated time dimension to the classic Tucker decomposition, they allow the model to learn a core tensor that balances invariant entity properties with time-specific transformations.
Taking this further, Jiaxin Pan and Steffen Staab proposed HGE (HGE: Embedding Temporal Knowledge Graphs in a Product Space of Heterogeneous Geometric Subspaces). Their model embeds information into a "product space" of different geometries (like Complex and Dual spaces). A relational attention mechanism then chooses the best geometry for the data: using one subspace to capture stable, hierarchical relationships and another to capture rapid, symmetric temporal shifts.
Why It Works
The success of these dual-path and gated models reveals a fundamental truth about intelligence: reasoning is the art of applying stable rules to a changing world. By explicitly modeling the "stable" (static facts) and the "changing" (temporal events) as distinct components, these systems achieve three things:
- Precision: They use static facts to prune the search space.
- Sensitivity: They prevent permanent data from masking new developments.
- Efficiency: They apply the right mathematical tool (sequential vs. set-based) to the right data type.