Disentangling Static and Dynamic Factors in Sequential Latent Spaces

This direction is split into other directions: spatiotemporal feature disentanglement. Its last version stays readable and citable here.

Version 1 of 1112 papers138 conceptsfrozen 2026-10-04

When a machine learning model observes a video of a person walking, it sees a complex stream of high-dimensional pixels. Within that stream, two fundamentally different types of information are intertwined: the static identity of the person (their height, clothing, and facial features) and the dynamic motion of the walk (the swing of the arms and the rhythm of the steps).

Standard sequence models often struggle because they conflate these factors. If a model represents a sequence as a single evolving path in latent space, it must "re-learn" the person's identity at every frame. This leads to "distractions"—where a speaker verification system focuses on the words being said rather than the voice itself, or a medical model confuses a patient's unique anatomy with the progression of a disease.

The research direction of Sequential Disentanglement addresses this by explicitly factorizing the latent space into two distinct components: a time-invariant "static" variable and a time-varying "dynamic" variable.

The Intuition of Factorization

The core hypothesis, as explored by Jen‐Tzung Chien in Contrastive Speaker Embedding With Sequential Disentanglement, is that identity-related features remain constant over time while content-related features vary. By isolating these, we create a "knob" for identity and a "knob" for motion.

Why does this work? Imagine a pendulum. Its motion is governed by its length (static) and its current angle (dynamic). If a model tries to predict the next position without knowing the length, it must infer the length from the motion every time. By separating the two, the model can use the static variable to "parameterize" the laws of physics, allowing the dynamic variable to focus solely on the state transition.

This principle is applied to Neural Ordinary Differential Equations (NODEs) by Ilze Amanda Auzina and Efstratios Gavves in Modulated Neural ODEs. They introduce "modulator" variables that represent time-invariant traits. This prevents the model from forcing all variations into the initial state, leading to much better long-term forecasting.

Mechanisms for Enforcing Separation

Simply having two latent variables isn't enough; the model will naturally try to leak information between them. Researchers use several clever constraints to ensure the static variable stays static and the dynamic variable stays dynamic:

  1. Temporal Alignment and Shuffling: In S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation, Martin Renqiang Min and colleagues use temporal shuffling as a "free" supervisory signal. If you shuffle the frames of a video, the identity (static) remains the same, but the motion (dynamic) is destroyed. This forces the static encoder to find features that are invariant to order.
  2. Orthogonality Constraints: To ensure the two spaces don't overlap, models often minimize the mutual information or enforce mathematical orthogonality between them. Guancheng Wan and Xiao Luo use this in Rethink GraphODE Generalization within Coupled Dynamical System to separate stable system attributes from time-varying states.
  3. Variance-Invariance Constraints: Ziyu Wang and Gus Xia propose in Emergent Interpretable Symbols and Content-Style Disentanglement via Variance-Invariance Constraints that content features should have high variance within a single sequence (because they change), while style features should have high variance across different sequences but remain constant within one.

Generalization and Robustness

The ultimate goal of this disentanglement is generalization. When a model understands that "identity" and "motion" are independent, it can handle scenarios it has never seen before—such as a known person performing a new action, or a new person performing a known action.

This is particularly critical in Evolving Domain Generalization (EDG). As Tiexin Qin and Haoliang Li demonstrate in Generalizing to Evolving Domains with Latent Structure-Aware Sequential Autoencoder, real-world data distributions (like sensor data or historical portraits) drift over time. By decomposing the latent space into time-invariant features and dynamic "shift" variables, the model can adapt to environmental changes without losing its grasp on the core task.

Applications Across Modalities

The power of this idea lies in its versatility. It is not limited to video or audio:

By treating "what stays the same" and "what changes" as mathematically distinct entities, these models move closer to a human-like understanding of the world, where identity is preserved even as the state of the world evolves.

Papers in this version

Showing 60 of 112 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_disentangling_static_and_dynamic_factors_9c8db507,
  title = {Disentangling Static and Dynamic Factors in Sequential Latent Spaces},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/disentangling-static-and-dynamic-factors-in-sequential-latent-spaces/rdr_9c8db507650b47a1ae631e08904afe6d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_disentangling_static_and_dynamic_factors_9c8db507_v1,
  title = {Disentangling Static and Dynamic Factors in Sequential Latent Spaces},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/disentangling-static-and-dynamic-factors-in-sequential-latent-spaces/rdr_9c8db507650b47a1ae631e08904afe6d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}