When modeling real-world entities—such as users in a marketplace, authors of scientific papers, or nodes in a social network—machine learning models face a fundamental tension known as the stability-plasticity dilemma.
Entities possess intrinsic, long-term traits that remain relatively invariant over time, such as a user's core personality, an author's native writing style, or a node's structural role in a global network. At the same time, these entities exhibit transient, time-varying behaviors: a user might temporarily browse for a gift outside their usual interests, an author might adopt a new research topic, or a node might experience a sudden burst of local interactions.
Traditional representation learning approaches typically fall into two extremes. They either treat entities as entirely static, which fails to capture behavioral drift, or they update embeddings continuously based on recent actions, which risks overfitting to ephemeral noise and suffering from catastrophic forgetting.
To resolve this, a prominent research direction focuses on static-dynamic factor decomposition. By explicitly separating an entity's latent representation into a stationary baseline and a time-varying correction term, models can simultaneously preserve long-term identity and adapt to short-term evolution.
The Mechanics of Decomposition
The core intuition behind static-dynamic decomposition is that any observed action is a joint product of who an entity fundamentally is and what context they are currently experiencing. Mathematically, instead of learning a single vector $\mathbf{h}_i(t)$ for an entity $i$ at time $t$, the architecture splits the representation into two distinct components:
$$\mathbf{h}_i(t) = f(\mathbf{s}_i, \mathbf{d}_i(t))$$
where $\mathbf{s}_i$ is a static embedding capturing invariant traits, $\mathbf{d}_i(t)$ is a dynamic state that evolves over time, and $f$ is a fusion mechanism (such as addition, concatenation, or gating).
┌────────────────────────┐
│ Observed Behavior │
└───────────▲────────────┘
│
[ Fusion Mechanism f ]
▲
┌───────────────┴───────────────┐
│ │
┌───────────────────────┐ ┌───────────────────────┐
│ Static Component │ │ Dynamic Component │
│ (Invariance) │ │ (Plasticity) │
└───────────────────────┘ └───────────────────────┘
▲ ▲
│ │
Global Attributes / Recurrent States /
Network Topology Temporal Sequences
This dual-component design changes how models learn from data. For instance, in collaborative filtering, Charlin et al. combine Poisson matrix factorization with Kalman filters to track latent factors. The static component establishes a user's baseline expertise or preference, while the dynamic component applies a time-varying correction. This prevents the system from overreacting to temporary excursions while remaining sensitive to genuine, long-term shifts in taste.
Similarly, Wu et al. implement this via Recurrent Recommender Networks (RRN), where stationary components capture persistent movie reputations and user traits, while dynamic components evolve via Long Short-Term Memory (LSTM) networks. When a user's behavior deviates from their historical norm, the dynamic component absorbs the transient shift, leaving the static anchor intact.
Why It Works: Balancing Stability and Plasticity
Decomposition works because it decouples the regularization requirements of long-term and short-term modeling.
1. Robustness Against Data Sparsity
In real-world data streams, interactions are often highly irregular. If an entity is inactive for a long period, a purely dynamic model loses track of its state or defaults to a global average. With a decomposed architecture, the static component provides a reliable fallback. As Park et al. demonstrate in dynamic graph node classification, the static aggregated view ensures structural stability when temporal snapshots are sparse, while the temporal view captures rapid behavioral shifts that aggregated models overlook.
2. Disentangling Sociability from Timing
An entity's behavior is governed both by how active it is overall and when it chooses to act. Xu et al. leverage this by forcing a unified set of embeddings to explain both an object's baseline "sociability" (its inherent likelihood of participating in an interaction) and its specific temporal dynamics. By separating the baseline propensity to interact from the timing of the events, the model learns features that are robust to the extreme sparsity typical of high-order interaction events.
3. Multi-Scale Temporal Modeling
The dynamic component itself can be further broken down to capture different temporal granularities. For example, Han et al. decompose entity representations in temporal knowledge graphs into a static embedding for inherent properties, a linear trend component for long-term evolution, and periodic seasonal functions to capture recurring patterns.
Application Across Domains
The strategy of fusing static structural baselines with dynamic sequence patterns manifests across several distinct domains.
Dynamic Graph and Social Network Integrity
In graph neural networks, the static component typically captures a node's global topological position, while the dynamic component tracks its local interaction sequences.
At Facebook, Noorshams et al. developed the TIES framework to detect malicious behavior by merging static social graph embeddings with dynamic embeddings derived from temporal action sequences. This hybrid approach leverages a fundamental insight: while bad actors can easily spoof short-term interaction sequences (dynamic), altering their fundamental placement within the broader social topology (static) is significantly harder.
Complementing this, Tu et al. show that combining static structural motifs (graphlets) with temporal sequence patterns provides a highly robust signature for network classification, proving that interaction order and underlying topology offer complementary signals.
Evolving Language and Text Modeling
In natural language processing, tracking the evolution of an author's style or a document's context requires a similar balance. Delasalles et al. utilize a dual-representation strategy within temporal language models, combining a static embedding for an author's invariant identity with a dynamic latent state that tracks topical drift over time. This allows the model to predict future text accurately, even when an author shifts genres or beats, because the core stylistic identity remains anchored.
Sequential Recommendation Systems
Modern recommenders frequently deploy pre-trained static embeddings alongside sequence-derived vectors. Wei et al. introduce a double-layer graph neural network that integrates dynamic purchase sequences with static global embeddings pre-trained via a Continuous Bag-of-Words approach. Similarly, Ghiye et al. maintain separate representations for permanent financial client characteristics and history-based behaviors, using personalized time-decay functions to govern how quickly the dynamic component reverts to the static baseline.
By formalizing the split between the permanent and the ephemeral, static-dynamic factor decomposition allows models across domains to navigate the tension between historical context and current trends, resulting in representations that are both stable and highly adaptive.
Key Foundational Work
Dynamic Poisson Factorization (2015) — contributed 2 concept elements to this direction.
Joint Training of Ratings and Reviews with Recurrent Recommender Networks (2017) — contributed 1 concept elements to this direction.
On the Formation of Circles in Co-authorship Networks (2015) — contributed 1 concept elements to this direction.