Many time series switch between a handful of distinct behaviors: a ball flying versus bouncing, a dancer's step types, a bee's waggle versus turn. Switching dynamical systems model this with a hidden discrete "regime" label that selects which dynamics govern the next time step. Recent theory shows these models are identifiable: the regimes and the hidden continuous trajectory can, in principle, be recovered uniquely from data. "Identifiable" here means that two different parameter settings cannot produce the same data distribution, up to relabeling and simple transformations. Every one of these guarantees, however, assumes the number of regimes $K$ is known in advance. Models that infer $K$, such as Dirichlet-process priors that create new states as needed, come with no identifiability guarantee. Practitioners therefore choose between a theory that needs $K$ and a method that finds $K$ without theory.
We propose to close this gap with the idea that resolves rotation ambiguity in sparse matrix factorization. If you fit a switching model with too many regime slots, the extra freedom does not take the form of arbitrary new behaviors. It takes the form of true regimes being split into copies that share the same dynamics, plus slots that are never used. Splitting a regime spreads its time share across several slots, which makes the occupancy vector "denser." Rotating a sparse matrix makes it denser in the same way. A penalty that rewards spiky occupancy therefore has a unique optimum at the true, minimal set of regimes. We will prove this in the population limit, give a consistent finite-sample version, and build it into a single exact-likelihood training run that replaces the usual sweep over $K$.
Core Claim
Take an identifiable switching dynamical system fitted with $K \ge K^*$ regime slots, where $K^*$ is the true number. We claim, and plan to prove, that every parameter setting reproducing the true observation distribution reduces to the true model once two operations are applied: empty slots are deleted, and slots with identical within-regime dynamics and emissions are merged. The redundancy of an overcomplete model is therefore fully described by "duplicate and leave empty." This is the analogue of the rotation group in factorization.
Let $\rho_k(\theta)$ be the long-run fraction of time spent in regime $k$ under parameters $\theta$. We train by maximizing
$$\frac{1}{n}\log p_\theta(x_{1:n}) \;+\; \lambda_n \sum_{k=1}^{K} \rho_k(\theta)^2 .$$
Here $x_{1:n}$ is the observed sequence and $\lambda_n$ is a penalty weight that shrinks with sample size $n$. Splitting a regime of mass $a+b$ into slots of mass $a$ and $b$ strictly lowers the penalty term, because $a^2+b^2<(a+b)^2$. The occupancy vector always sums to one, so an $\ell_1$ penalty would be constant and useless. What we need is a spikiness reward, in the same spirit as the $\ell_4$ maximization of Unique sparse decomposition of low rank matrices, which recovers sparse factors by maximizing spikiness on the sphere. The population optimum is the merged, minimal solution. We expect consistency in $K$ when $\lambda_n \to 0$ slowly enough to dominate the $O(n^{-1/2})$ statistical fluctuation of the likelihood, giving a BIC-like rate. If the true model has a regime with tiny occupancy, a fixed $\lambda$ will merge it away, and the theory must quantify that tradeoff honestly.
Relation to Prior Work
The fixed-$K$ identifiability results supply our starting point. On the Identifiability of Markov Switching Models treats the model as a finite mixture over trajectories and shows that regimes with real-analytic transition functions are identifiable up to permutation. On the Identifiability of Switching Dynamical Systems extends this to nonlinear decoders, recovering latents up to affine maps. End-to-End Identifiable and Consistent Recurrent Switching Dynamical Systems handles switches that depend on the continuous state and trains with exact likelihood using normalizing flows (invertible networks with tractable densities). Identifiable Markov Switching Models with Instantaneous Effects and Exponential Families adds non-Gaussian flow emissions. All four freeze $K$. We reuse their mixture-over-trajectories argument to characterize what happens when $K$ is too large, which none of them analyze. The duplication case is subtle because the hidden chain must only be lumpable in a weak sense, not split trivially, and this is where the new proof work lies.
The nonparametric line motivates the problem and is the main competitor. Nonparametric Bayesian Learning of Switching Linear Dynamical Systems infers $K$ with a sticky hierarchical Dirichlet process. Disentangled sticky hierarchical Dirichlet process hidden Markov model separates per-state persistence from transition similarity. Parsing neural dynamics with infinite recurrent switching linear dynamical systems lets states emerge over time under a distance-dependent prior. None proves that the recovered regimes are unique. All are known to spawn spurious states when emission noise is misspecified, for instance heavy tails mistaken for new regimes. We keep exact maximum-likelihood training with flexible emissions and move cardinality selection into a penalty with a uniqueness proof.
The sparsity line supplies the mechanism and also a constraint. On the Identifiability of Sparse ICA without Assuming Non-Gaussianity and On the Identifiability of Nonlinear ICA: Sparsity and Beyond show that structural sparsity removes coordinate ambiguity in static mixing. We transfer that logic from mixing coefficients to regime occupancy. A Regularized Linear Dynamical System Framework for Multivariate Time Series Analysis is the closest operational relative: it overparameterizes the continuous state dimension of a single-regime linear system and prunes it with group and nuclear-norm penalties. That work never asks whether the pruned solution is unique. It also cannot prune discrete regimes, whose redundancy has a combinatorial "duplicate" structure that row sparsity does not see.
Research Program
The theory has three parts. First, we will characterize the equivalence class of overcomplete switching models, for the Markov-switching case and then the recurrent case, under the real-analyticity and weak-injectivity assumptions already in use. Second, we will show that strictly convex occupancy rewards select a unique representative, the true model up to permutation. Third, we will give a finite-sample consistency statement for $\hat K$ that states explicitly how small an occupancy, or how close two regimes' dynamics, can be resolved at sample size $n$.
The algorithm adds the penalty to the M-step of the exact-likelihood EM used by the recurrent flow-based model. Occupancies are computed from posterior state counts, so the penalty acts on quantities the E-step already produces. Slots that become empty stay available. In streaming or drifting data, a new behavior can claim an empty slot instead of being forced into an old regime. This gives an online-capable alternative to nonparametric growth.
A secondary analysis targets near-duplicates. In finite samples, split copies differ slightly, so their masses may not fully concentrate. We will test whether the occupancy reward alone merges them, or whether a fusion penalty on differences between regime dynamics is also needed.
Baselines
The naive reference is the same identifiable flow-based switching model run with $K$ deliberately too large and no penalty. A variant adds post hoc removal of regimes below an occupancy threshold. This shows what overparameterization does by itself.
The standard practitioner baseline is a fixed-$K$ identifiable model, following End-to-End Identifiable and Consistent Recurrent Switching Dynamical Systems or the Markov-switching estimator of On the Identifiability of Markov Switching Models, swept over $K$ and selected by BIC and by held-out likelihood. To keep the comparison fair, the sweep receives the same total compute as our single run plus its $\lambda$ search, rather than unlimited restarts. For reference, the file reports a synthetic latent mean correlation coefficient of about 0.99 and a dance-video pixel MSE of $2.26\cdot10^{-4}$ for the identifiable Markov-switching model at known $K$ (On the Identifiability of Markov Switching Models). On non-Gaussian synthetic data, FlowMSM reports normalized mutual information above 0.80 (Identifiable Markov Switching Models with Instantaneous Effects and Exponential Families). Beating this baseline means matching these oracle-$K$ numbers without being told $K$, at a fraction of the sweep's compute.
The strongest existing method for finding $K$ is the nonparametric family: sticky HDP-SLDS, DS-HDP-HMM, and irSLDS. We run their published samplers or code on identical data with a matched wall-clock budget. No reported result was found on Lacuna for numeric segmentation or likelihood scores of sticky HDP-SLDS on honey bees or IBOVESPA, or of irSLDS on NASCAR or IBL. The file reports only qualitative wins, so we will rerun them ourselves. Beating them means more frequent exact recovery of $K$, higher latent correlation and regime agreement, and fewer spurious states under heavy-tailed noise.
Two ablations isolate the key ingredient. The first replaces the squared-occupancy reward with a log penalty. This is equivalent to MAP estimation under a sparse Dirichlet prior on regime weights and tests whether the specific concentration function matters. The second applies our penalty to a variational switching model that is not identifiable, which tests whether the guarantee depends on identifiability of the underlying model as we claim.
Why not just…?
Why not just sweep $K$ and pick by BIC? This is a real competitor on stationary data, and BIC may well match us in $K$ accuracy there. Its weaknesses are cost and drift. It requires one full fit per candidate. It also cannot add a regime mid-stream, which is exactly the failure that Parsing neural dynamics with infinite recurrent switching linear dynamical systems documents for fixed-$K$ recurrent switching models on the extended NASCAR task. If BIC equals us on stationary benchmarks at equal compute, we will report that and restrict our claim to the compute and drift settings.
Why not just use a sticky HDP-SLDS or DS-HDP-HMM? These models do find $K$, but there is no proof that what they find is unique. Their Gaussian conjugate emissions also push them to split heavy-tailed regimes into spurious states. Our heavy-tail synthetic experiment tests this directly, with true $K^*$ fixed and tail thickness varied. If HDP models stay at $K^*$ while noise kurtosis grows, this motivation weakens.
Why not just put a sparse Dirichlet prior on the weights of an overcomplete fixed-$K$ model? This is the most serious threat, because a log penalty on occupancy is exactly that. We do not claim the penalty's form is new. Our contribution is the theorem that the overcomplete equivalence class of an identifiable switching system consists only of duplicates and empty slots, which is what turns any strictly subadditive occupancy penalty into a uniqueness guarantee. The log-versus-squared ablation will show whether the choice of penalty matters in practice. If it does not, we will present the Dirichlet form as the recommended implementation.
Why not just train overcomplete and drop rarely used regimes? A split copy can carry half of a regime's mass, so no threshold removes it without also removing rare true regimes. The duplicated-initialization test below directly measures this failure.
Proposed Experiments
On synthetic Markov-switching and recurrent switching systems, $K^*$ ranges from 3 to 8 and we fit $K=2K^*$ and $K=20$. We vary sample length, the occupancy of the rarest regime (down to 2%), how similar the regimes' dynamics are, and emission tail thickness. The metrics are exact-$K$ recovery rate, mean correlation coefficient of the recovered latents, regime agreement measured by Hamming distance and normalized mutual information, and held-out log-likelihood. The real benchmarks are bouncing-ball videos and AIST dance videos, which allow comparison to the identifiable baselines' reported forecasting and reconstruction; honey bee trajectories, which have labeled modes for Hamming distance; NASCAR and extended NASCAR, for regime emergence; and IBL Neuropixels recordings, for held-out likelihood under slow drift.
The decisive checks are three. First, initialize at the true solution with one regime duplicated and its mass split evenly. The penalized objective should merge the copies, while the threshold and unpenalized baselines keep both. Second, sweep the rarest regime's occupancy against $\lambda$ to map the boundary predicted by the finite-sample theory; if empirical failures fall far from it, the theory is wrong. Third, on extended NASCAR, the regime that appears midway should claim an empty slot, matching irSLDS on regime detection while also recovering latents with identifiable-model accuracy.
References
- Unique sparse decomposition of low rank matrices
- On the Identifiability of Markov Switching Models
- On the Identifiability of Switching Dynamical Systems
- End-to-End Identifiable and Consistent Recurrent Switching Dynamical Systems
- Identifiable Markov Switching Models with Instantaneous Effects and Exponential Families
- Nonparametric Bayesian Learning of Switching Linear Dynamical Systems
- Disentangled Sticky Hierarchical Dirichlet Process Hidden Markov Model
- Parsing neural dynamics with infinite recurrent switching linear dynamical systems
- On the Identifiability of Sparse ICA without Assuming Non-Gaussianity
- On the Identifiability of Nonlinear ICA: Sparsity and Beyond
- A Regularized Linear Dynamical System Framework for Multivariate Time Series Analysis