Dynamic Latent State Discovery in Sequential Models

66 concepts65 papers8 proposals

When modeling time-series data—whether tracking the progression of a disease, segmenting animal behavior, or decoding financial market regimes—we often rely on latent variable frameworks like Hidden Markov Models (HMMs) or Partially Observable Markov Decision Processes (POMDPs). These models assume that the observed data is driven by an underlying, unobserved "state."

However, traditional formulations suffer from a fundamental constraint: the designer must pre-specify the exact number of hidden states (the state cardinality). This creates a fragile trade-off. Choose too few states, and the model underfits, failing to capture subtle but critical dynamics. Choose too many, and the model overfits, capturing environmental noise or creating redundant parameters. To escape this dilemma, a prominent research direction focuses on dynamic latent state discovery: building models where the state cardinality is not a fixed hyperparameter, but a variable inferred directly from the data.

The Goldilocks Dilemma of Fixed Cardinality

In sequential modeling, hidden states serve as a compact summary of history. In a fixed-cardinality model, this summary is bounded by a hard limit. As noted in work by Hao Zhu et al. in Multi-intention Inverse Q-learning for Interpretable Behavior Representation, forcing a fixed number of states limits a system's ability to adapt to unknown behavioral or environmental complexity without manual intervention.

Traditionally, finding the "right" number of states requires exhaustive model selection. Practitioners train multiple separate models with different state counts ($K=2, 3, 4\dots$) and compare them using information criteria like Akaike’s Information Criterion (AIC) or the Bayesian Information Criterion (BIC). For instance, Eiji Uchibe et al. utilized AIC to determine state-space dimensionality for robotic behavior acquisition in State Space Construction for Behavior Acquisition in Multi Agent Environments with Vision and Action. Similarly, Jie Ding et al. developed data-driven metrics to optimize state counts for autoregressive models in Data-Driven Learning of the Number of States in Multi-State Autoregressive Models.

While information criteria work for static datasets, they fail in dynamic, streaming, or highly heterogeneous environments. If a model encounters a completely new regime midway through a data stream, a fixed-cardinality architecture cannot allocate a new state to represent it.

Nonparametric Priors for Infinite State Spaces

Instead of sweeping through fixed values of $K$, why not let $K \to \infty$ theoretically, but let the data bound it practically? This is the core intuition behind nonparametric Bayesian sequence models.

In the foundational paper The Infinite Hidden Markov Model, Matthew J. Beal et al. introduced the Infinite Hidden Markov Model (iHMM). As later summarized by Zoubin Ghahramani in (Invited Talk) Bayesian Hidden Markov Models and Extensions, the iHMM replaces the standard fixed-size transition matrix with a Hierarchical Dirichlet Process (HDP).

The HDP acts as a self-regulating mechanism. The model begins with a small number of active states. When it encounters an observation that deviates significantly from what the current states expect, the HDP prior allows the model to spawn a brand-new state with a small probability. Crucially, the prior penalizes the creation of new states, ensuring that the state space only expands when there is overwhelming evidence in the data.

This paradigm extends across various sequential decision-making and structural frameworks:

State Reusability and Change-Point Detection

A naive model might handle novelty by simply creating a new state every time the data distribution shifts. However, real-world systems are often cyclical or recurrent. A financial market might return to a "bear market" regime seen years ago; an animal might alternate between "foraging" and "resting" states. A robust model must not only discover new states but also recognize when to reuse old ones.

To achieve this, researchers leverage the Chinese Restaurant Process (CRP) prior. In Bayesian Inference for Change Points in Dynamical Systems with Reusable States - a Chinese Restaurant Process Approach, Florian Stimberg et al. demonstrated how a CRP prior allows a system to detect abrupt change-points and assign the post-change dynamics to either a brand-new state or an existing, previously archived state.

This balance of novelty and reusability is central to tracking non-stationary data streams without state explosion, a property heavily utilized in hierarchical change-point detection (Pablo Moreno-Muñoz in Continual Learning for Infinite Hierarchical Change-Point Detection) and multidimensional change-surface characterization (Andrew Gordon Wilson et al. in Scalable Gaussian Processes for Characterizing Multidimensional Change Surfaces).

Making Infinity Computable

While infinite state spaces are mathematically elegant, they introduce a massive computational hurdle: calculating a posterior distribution over an infinite parameter space is analytically intractable. Early Markov Chain Monte Carlo (MCMC) sampling methods were slow and prone to getting stuck in local modes due to the "sticky" nature of sequential dependencies.

To make dynamic state discovery practical, the field relies on three core algorithmic innovations:

  1. Beam Sampling: In Beam sampling for the infinite hidden Markov model, Jurgen Van Gael et al. combined slice sampling with dynamic programming. By introducing an auxiliary variable that adaptively truncates the infinite state space to a finite, manageable "beam" of likely states at each time step, they enabled the efficient sampling of entire state trajectories simultaneously.
  2. Particle Gibbs: For complex or non-linear systems, Nilesh Tripuraneni et al. proposed Particle Gibbs for Infinite Hidden Markov Models. This method uses a "lazy" state expansion strategy alongside ancestor sampling, allowing the inference process to scale smoothly as new states are discovered.
  3. Scalable Variational Inference: When datasets scale to millions of observations, sampling methods become too slow. To address this, Michael C Hughes et al. developed a variational framework in Scalable Adaptation of State Complexity for Nonparametric Hidden Markov Models that utilizes explicit "birth, merge, and delete" proposals. This allows the model to deterministically optimize its state complexity. Similarly, Aonan Zhang introduced Stochastic Variational Inference for the HDP-HMM, enabling closed-form parameter updates that scale to massive sequential datasets.

Optimization-Based Alternatives

Nonparametric Bayesian methods approach dynamic cardinality from the bottom up, allowing a small model to grow as needed. An alternative, non-Bayesian approach tackles the problem from the top down: start with an excessively large state space and use optimization techniques to prune away unneeded dimensions.

In A Regularized Linear Dynamical System Framework for Multivariate Time Series Analysis, Zitao Liu 0003 et al. introduced a Regularized Linear Dynamical System (rLDS). Instead of relying on Dirichlet priors, the authors apply sparsity-inducing penalties (such as $L_1$ or group-lasso norms) directly to the system's transition matrix. During training, the optimization objective drives the transition probabilities of redundant or unnecessary hidden states to absolute zero, effectively shrinking the active state space to its optimal cardinality.

Ultimately, whether through the probabilistic growth of nonparametric priors or the structured pruning of sparsity regularization, moving away from fixed state cardinality allows sequential models to mirror the open-ended complexity of the real world.

Key Foundational Work

The Infinite Hidden Markov Model (2001) — contributed 1 concept elements to this direction.
State Space Construction for Behavior Acquisition in Multi Agent Environments with Vision and Action (1998) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_fixed_state_cardinality_60023dd8,
  title = {fixed state cardinality},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fixed-state-cardinality/rdr_60023dd8b8244544b61eee89adfc98b0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_fixed_state_cardinality_60023dd8_v1,
  title = {fixed state cardinality},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fixed-state-cardinality/rdr_60023dd8b8244544b61eee89adfc98b0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake