Many time series switch between regimes. An animal grooms, then runs. A neural population idles, then engages in a task. A market is calm, then volatile. Models that segment such data usually make two choices in advance. They fix how many regimes exist, and they assume every regime moves within a hidden space of the same size. Bayesian nonparametric methods (models whose number of components grows with the data instead of being set beforehand) have largely removed the first assumption for sequences. The second has survived intact. A low-dimensional regime, such as a steady stereotyped movement, is forced to share the latent size of the most complex regime in the dataset. This wastes parameters, blurs the boundaries between regimes, and hides a scientifically interesting quantity: how complex each regime actually is.
We propose a single deterministic objective that segments a multivariate time series into an unbounded number of reusable regimes. Each regime is a low-dimensional linear subspace of its own learned dimension, with its own linear dynamics inside that subspace. The objective comes from taking the zero-noise limit of a Bayesian model. That limit turns the model's prior into a few explicit penalties: one for each switch between regimes, one for each regime that exists, and one for each dimension a regime uses. Coordinate descent on this objective is fast and can run on a stream. It creates a new regime only when doing so lowers the cost, and it grows or shrinks each regime's dimension in the same way. The method joins two existing lines of work that have never been combined: zero-noise limits of infinite hidden Markov models, which handle temporal regimes but use flat Gaussian states, and zero-noise limits of nonparametric subspace clustering, which handle variable-dimension subspaces but ignore time.
Core Claim
The approach rests on two prior results. Small-Variance Asymptotics for Hidden Markov Models showed that as observation noise shrinks, the infinite hidden Markov model becomes a hard-assignment objective. That objective combines squared error with three penalties: one on state switches, one on the number of states reachable from each state, and one on the total number of states. The resulting algorithm matched a 1,000-sample beam sampler at the cost of about 10 samples. DP-space took the same limit for a Dirichlet-process mixture of PCA models. It obtained a cost of $\lambda K + s\sum_k d_k$ plus squared distance to each assigned subspace, which selects both the cluster count and per-cluster dimensions $d_k$. It reported about 3.3% error on Hopkins 155 in about 2 seconds, against more than 500 seconds for sparse subspace clustering. But DP-space treats points as independent, and asymp-iHMM's states are single points in observation space.
Our claim is that these limits compose. Consider a sticky hierarchical-Dirichlet-process switching model in which regime $k$ has a mean $\mu_k$, an orthonormal basis $U_k$ with $d_k$ columns, and a $d_k \times d_k$ dynamics matrix $A_k$. Let both off-subspace and dynamics noise shrink to zero at the same rate. The result is the objective
$$ \min_{z,\{\mu_k,U_k,A_k,d_k\},K}\ \sum_{t}\Big(\|r^{\perp}_t\|^2+\|r^{\mathrm{dyn}}_t\|^2\Big)+\lambda_1\sum_t \mathbb{I}(z_t\neq z_{t-1})+\lambda_2\sum_j|\{k:N_{jk}>0\}|+\lambda_3 K+s\sum_k d_k . $$
Here $z_t$ is the regime at time $t$. The term $r^{\perp}_t=(I-U_{z_t}U_{z_t}^\top)(y_t-\mu_{z_t})$ is the part of observation $y_t$ that lies off its regime's subspace. The term $r^{\mathrm{dyn}}_t$ is the one-step prediction error of the in-subspace coordinates $c_t=U_{z_t}^\top(y_t-\mu_{z_t})$ under $A_{z_t}$; it is computed only when $z_t=z_{t-1}$, so that coordinates from different regimes are never compared. $N_{jk}$ counts transitions from regime $j$ to regime $k$, and $s$ is the price of one latent dimension. The novelty is the joint presence of temporal penalties, reusable regimes, and per-regime dimension in one cost that decreases monotonically under coordinate descent. It is not any single one of these elements.
Research Program
The first deliverable is the derivation. We will state the generative model and show which hyperparameter scalings produce each penalty in the limit. We will also identify where the derivation needs care. Prediction terms only within segments make the cost non-Markov in a mild way, and the dimension prior must be scaled so that $s$ survives the limit, as the truncated geometric prior does in DP-space.
The second deliverable is the algorithm. It alternates four steps. The first is a dynamic-programming pass over $z$ given the regimes, using local birth and merge moves in the style of asymp-iHMM. A new regime starts as a short segment whose cost, including $\lambda_3$, is lower than any existing assignment. The second step refits each regime's subspace by truncated SVD. Its dimension is chosen by the DP-space rule: add a dimension when the residual energy it captures exceeds $s$. The third step refits $A_k$ by least squares on in-subspace coordinates. The fourth step is a merge sweep that joins two regimes when the switching and regime-count penalties saved exceed the added residual. Each step does not increase the objective. We will prove this, as DP-space did for its static case.
The third deliverable is a streaming variant. It keeps sufficient statistics per regime (running means, scatter matrices, cross-covariances) and decides to create a new regime online. This addresses the main weakness of fixed-cardinality models in Section 1 of the motivating literature: they cannot allocate a regime they have never seen without retraining.
The known weakness of zero-noise methods is that they swap "choose $K$" for "choose penalties." We treat this as a central question, not a footnote. The penalties are reparameterized in units of the median per-step residual of a single global PCA model, which makes them scale-free. All four are then chosen by one held-out next-step prediction criterion on a validation segment. We will report full sensitivity surfaces, so readers can see whether we have only moved the tuning burden.
Baselines
The naive reference point is a fixed-cardinality switching linear dynamical system. It is swept over $K$ and a single global latent dimension $D$, and the pair is chosen by BIC. This is the procedure practitioners actually use. It gets the same data and a wall-clock budget equal to the total time of all fits in the sweep. Beating it means higher segmentation accuracy, or comparable accuracy in less total time, on data whose regimes differ in dimension.
The strongest nonparametric competitor for segmentation is the sticky HDP-HMM. The Sticky HDP-HMM adds a self-transition bias to stop the plain model from jittering between duplicate states. We pair it with its continuous-state relative, the HDP-SLDS from Nonparametric Bayesian Learning of Switching Linear Dynamical Systems, which infers the number of linear modes but uses a fixed latent dimension for all of them. On the honeybee dance benchmark, RED-HDP-HMM reports 0.873 segmentation accuracy for the sticky HDP-HMM and 0.899 for RED-HDP-HMM itself, the best reported number there. Bayesian Inference in Recurrent Explicit Duration Switching Linear Dynamical Systems reports about 0.85 for its fixed-$K$ explicit-duration model and about 0.37 for the recurrent SLDS. For speed, the reference is asymp-iHMM's reported parity with a 1,000-sample beam sampler at roughly the cost of 10 samples. On S&P 500 tracking, asymp-iHMM reports a mean absolute percentage error of 2.78% against 6.44% for a parametric HMM.
Sampler baselines run with their published hyperparameter defaults plus the same validation-based tuning budget, measured in model fits, that our penalties receive. Each is given the same wall-clock time, and we also report accuracy at convergence. Our method has no explicit duration model, so we do not expect to beat 0.899 on honeybee. Beating the sticky HDP-HMM's 0.873 at a fraction of its runtime would count as success there. The comparison that matters most is against HDP-SLDS on data with unequal regime dimensions.
Two ablations isolate the key ingredient. The first ties all $d_k$ to one global value, selected by the same validation criterion. The second removes the temporal penalties, which reduces the method to DP-space on individual frames. If tying $d_k$ costs nothing, the core claim is wrong.
Why not just…?
Why not just run DP-space on frames and smooth the labels with an HMM afterwards? This two-stage pipeline shares our ingredients but fits subspaces without knowing about time. Each subspace is then estimated from frames that the later smoothing would have relabeled, and transient frames seed spurious clusters. The asymp-iHMM derivation suggests why this is suboptimal: its switching penalty changes which states exist, not only which labels survive smoothing. We run this pipeline as an explicit baseline. If it matches joint optimization, the joint objective is not justified and we will say so.
Why not just use HDP-SLDS with a large latent dimension and a shrinkage prior on each mode's emission columns? This is the real threat. Per-mode automatic relevance determination (a prior that drives unneeded latent dimensions toward zero) could plausibly mimic mode-specific dimension inside a sampler. Nothing in our sources shows it failing. Its likely weaknesses are cost and memory: Nonparametric Bayesian Learning of Switching Linear Dynamical Systems notes that its Gibbs sampler slows as latent dimension grows, and shrinkage does not remove dimensions from the computation. Its interpretability may also be weaker, because soft shrinkage gives no crisp $d_k$. We will implement it and compare recovery of the true $d_k$ and accuracy at matched wall-clock time on the synthetic suite. If it matches us on accuracy, our remaining contribution is speed and streaming. The note would then be rewritten around those claims.
Why not just run asymp-iHMM on PCA-reduced data? Global PCA picks one projection for all regimes. A low-variance but distinctive regime, such as a small stereotyped movement, may be discarded entirely. Asymp-iHMM's flat Gaussian states also cannot represent within-regime linear dynamics. The synthetic suite deliberately includes a low-variance, low-dimensional regime to test this.
Why not just sweep $K$ and $D$ with BIC? The two-level search over $K$ and a dimension per regime is combinatorial, which is the problem DP-space was built to avoid. A sweep with one shared $D$ is cheap but cannot express unequal dimensions. Sweeping is also a batch procedure and cannot add a regime partway through a stream. Parsing neural dynamics with infinite recurrent switching linear dynamical systems reports that fixed-$K$ recurrent SLDS "struggled by trying to cram new behaviors into its old, fixed categories" on the extended NASCAR task. No number is reported there, so we will produce one.
Proposed Experiments
The decisive synthetic test is an extended NASCAR variant: a 2D switching trajectory of the kind used by the irSLDS and REDSLDS papers, embedded linearly in 30 observed dimensions. Partway through the sequence, new regimes appear whose dynamics occupy 1, 3, or 8 latent dimensions, and one existing regime reappears. Metrics are segmentation accuracy after Hungarian matching, exact recovery of $K$ and of each $d_k$, held-out next-step prediction error, and wall-clock time. For the streaming variant we also measure the delay between a regime's onset and the creation of a new state. The prediction is that the tied-dimension ablation and HDP-SLDS will either split the 8-dimensional regime or merge the 1-dimensional regimes, and that the full method will not.
On real data, honeybee dance tests whether the method holds its ground on low-dimensional regimes against the reported 0.873, 0.85, and 0.899. The C. elegans calcium recordings used by RED-HDP-HMM, where it reports gains of 4.5 to 10.4 points over the sticky HDP-HMM, test neural regimes. For IBL Neuropixels recordings, no reported number was found on Lacuna. The scientific check there is whether inferred $d_k$ differ systematically between task-engaged and disengaged epochs, and whether held-out prediction beats tied dimensions. For BehaveNet mouse behavior, likewise no reported number was found on Lacuna; we compare held-out prediction and segment stability against HDP-SLDS. As a sanity check, setting the switching penalty to zero on Hopkins 155 should reproduce DP-space's roughly 3.3% error. Rerunning S&P 500 tracking should match asymp-iHMM's 2.78%, since one-dimensional data leaves no room for variable dimension.
The project fails cleanly if either of two things happens. One is that per-regime dimension gives no gain over the tied-dimension ablation on the variable-dimension synthetic task. The other is that the shrinkage-based HDP-SLDS matches the full method at equal wall-clock time while also recovering the true dimensions. Penalty sensitivity surfaces are reported for every dataset. If accuracy within a factor-of-two band of penalties varies more than the gap to the sticky HDP-HMM, the method has not solved the model-selection problem. It has only relocated it.
References
- Small-Variance Asymptotics for Hidden Markov Models
- DP-space: Bayesian Nonparametric Subspace Clustering with Small-variance Asymptotics
- The Sticky HDP-HMM: Bayesian Nonparametric Hidden Markov Models with Persistent States
- Nonparametric Bayesian Learning of Switching Linear Dynamical Systems
- RED-HDP-HMM: Observation-Dependent Durations for Bayesian Nonparametric Sequential Models
- Bayesian Inference in Recurrent Explicit Duration Switching Linear Dynamical Systems
- Parsing neural dynamics with infinite recurrent switching linear dynamical systems