Pruning State-Space Models for the Gaps They Will Actually See

State-space models (SSMs) are sequence networks whose hidden memory is a set of decaying "modes," each a number that is multiplied by a factor at every step. They are attractive for irregularly sampled data such as ICU records, where lab values and vital signs arrive at uneven, asynchronous times. For such data the model can be fed the elapsed time between observations directly: a mode with continuous decay rate $\lambda$ is multiplied by $e^{\lambda \Delta t}$ across a gap of length $\Delta t$. Compression tools for these models decide which modes to keep, either in a single pass after training or during training. They score modes as if the model ran on a regular grid with a fixed step. That assumption is harmless for text, but wrong for a patient who is measured every few minutes during a crisis and every few hours otherwise. Under long gaps, a fast-decaying mode forgets everything before the next observation arrives. Under bursts of closely spaced readings, the same mode can be the one that tracks the deterioration.

We propose a one-shot, post-training method that removes SSM state dimensions according to their importance under the observed distribution of time gaps. The method extends balanced truncation, the control-theory procedure that ranks a linear system's internal states by how much energy flows through them from input to output, from fixed-step systems to systems sampled at random times. The central observation is simple. For a diagonal SSM with independent gaps, the quantities balanced truncation needs depend on the gaps only through their moment generating function evaluated at the model's own decay rates. Two consequences follow. First, the gap distribution can be measured from timestamps alone, without labels. Second, a model trained once can be pruned differently for different sampling regimes, for example an ICU versus a general ward.

Core Claim

Existing compression of SSMs uses fixed-step importance, and that importance is biased in a predictable direction when the time between observations varies. COMPRESSM, from The Curious Case of In-Training Compression of State Space Models, truncates low-energy state dimensions early in training. For time-varying dynamics it uses a "mean LTI surrogate," a single fixed system obtained by averaging the varying dynamics. Hankel Singular Value Regularization for Highly Compressible State Space Models penalizes the same energy spectrum during training and states explicitly that it covers only time-invariant layers.

Averaging the step is the natural surrogate, and it misranks modes. Because $e^{2\,\mathrm{Re}(\lambda)\Delta t}$ is convex in $\Delta t$, Jensen's inequality gives

$$\mathbb{E}\big[e^{2\,\mathrm{Re}(\lambda)\,\Delta t}\big] \;\ge\; e^{2\,\mathrm{Re}(\lambda)\,\mathbb{E}[\Delta t]},$$

where the expectation is over the gap distribution. The gap between the two sides grows with how bursty the sampling is and with how fast the mode decays. A mean-step surrogate therefore systematically undervalues fast modes on bursty clinical data, which is exactly where acute events live. On data with few bursts, the error goes the other way: the surrogate can keep fast modes that forget everything between sparse observations.

Our claim is that ranking modes by the correct expectation gives better accuracy at equal state size. We also claim that the set of modes worth keeping shifts in a direction we can predict as sampling density and burstiness change. Both claims can be tested without retraining.

Method

Consider a diagonal SSM layer with continuous eigenvalues $\lambda_i$ (negative real part), input vectors $b_i$, and output vectors $c_i$. Each observation $k$ arrives after gap $\Delta t_k$. The state update is $x_{i,k} = e^{\lambda_i \Delta t_k} x_{i,k-1} + \beta_i(\Delta t_k)\, b_i^\top u_k$, where $\beta_i$ is the zero-order-hold input gain.

Assume the gaps are independent and identically distributed, and independent of the inputs. Then the stationary controllability Gramian, which measures how much input energy reaches each pair of modes, has entries

$$P_{ij} = \frac{\mathbb{E}\big[\beta_i(\Delta t)\,\beta_j^*(\Delta t)\big]\; b_i^\top \Sigma_u\, b_j^{*}}{1 - \varphi(\lambda_i + \lambda_j^*)}, \qquad \varphi(s) = \mathbb{E}\big[e^{s\,\Delta t}\big].$$

Here $\Sigma_u$ is the input covariance and $\varphi$ is the moment generating function of the gaps. The observability Gramian, which measures how much each mode's energy reaches the output, is built the same way from the $c_i$. The square roots of the eigenvalues of their product are the "sampled" Hankel singular values. Truncating to the top modes is ordinary balanced truncation of this jump-sampled system.

Estimating $\varphi$ requires only the timestamps of a calibration set. When input values are expected to correlate with gap length, we use a second, joint estimator that averages $e^{(\lambda_i+\lambda_j^*)\Delta t_k}\,\beta_i\beta_j^*\,(b_i^\top u_k)(b_j^\top u_k)^*$ directly over calibration sequences. This matters on PhysioNet, where clinicians order tests because the patient is changing, so the time of a measurement is itself informative. After truncation, an optional single least-squares refit of the output projection on calibration activations corrects residual error. This follows the layer-wise reconstruction idea used in one-shot pruning, and no gradient training is involved.

The scope is deliberately narrow. We handle SSMs whose only time variation is the physical gap $\Delta t_k$. We do not handle fully input-selective layers like Mamba, where the mean-surrogate problem compounds with selection and no closed form exists. Feedforward weights in these models are pruned with an off-the-shelf method in every arm of the comparison, so differences come only from the treatment of the state.

Relation to Prior Work

Three groups of papers matter here, in different roles.

Two papers motivate the work by showing that recurrence breaks generic pruning. SparseSSM adapts the second-order Optimal Brain Surgeon procedure to Mamba. It shows that magnitude pruning and SparseGPT collapse beyond roughly 30% sparsity because errors accumulate through the shared transition matrix. It also weights early tokens more heavily through a power-law prefix scheme. That scheme assumes importance follows sequence position, which conflicts with clinical data where decisive bursts occur deep into a stay; we test this directly. One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation scores weights by how they preserve the recurrent Jacobian over time and reports that unnormalized gradient saliency fails (about 89% error on sequential MNIST). Both papers work on fixed-step data.

Three papers supply the method and the strongest competitors. COMPRESSM and HSVR, cited above, provide the balanced-truncation machinery. COMPRESSM also provides the evidence that pruning a large model beats training small: 85.7% versus 81.8% on LRA CIFAR-10 at about 92 state dimensions. We reuse their energy criterion and replace the fixed-step or mean-step assumption with the gap distribution. Pruning via Sparsity-indexed ODE also uses an ODE, but its continuous variable is the sparsity level of a vision network, not physical time. It is related in name only.

Four papers set the application regime and its constraints. Neural Controlled Differential Equations for Irregular Time Series and ContiFormer show that irregular spacing changes latent dynamics, and that continuous-time models handling it are slow: Neural CDEs run about 5x slower than RNNs. Neither compresses anything. From 805 ms to 23 ms shows that preprocessing irregular data can account for more than 85% of latency. That constrains what speedup claims we can make. On Pruning State-Space LLMs warns that structured pruning of SSMs is fragile, with head pruning degrading sharply at 25%. Our structured unit is therefore a state mode, not a head.

Research Program

We train dense diagonal SSM classifiers with large state size on each benchmark. Each model receives per-observation gaps and observation-mask channels. On the synthetic benchmark, we train under several observation-drop rates. We then compress each trained model one time per criterion and per target state fraction (75%, 50%, 25%, 10% of modes kept) and evaluate without retraining.

The analysis has three parts. First, we test the Jensen prediction: on bursty versus regular synthetic gap processes, we compare which modes, indexed by decay rate, each criterion keeps. Second, we measure how accuracy depends on mismatch between the calibration sampling regime and the deployment regime. Third, we check whether informative missingness breaks the independence estimator and whether the joint estimator repairs it.

Baselines

The naive reference is the dense model, together with two cheap criteria under the same state budgets: random mode removal, and removal of the fastest-decaying modes (largest $|\mathrm{Re}\,\lambda|$). The second encodes the folk view that slow modes carry long-range memory, and it costs nothing. Beating it at moderate sparsity is the minimum bar. If our criterion merely reproduces this ordering, the gap-distribution machinery is unnecessary.

Two baselines differ from our method in exactly one respect and form the key ablations. One applies COMPRESSM-style balanced truncation The Curious Case of In-Training Compression of State Space Models post hoc to the same trained weights, using the mean-gap surrogate. It isolates the effect of using the full gap distribution versus its mean. The other uses a fixed unit step, the regime under which COMPRESSM and HSVR were developed. All three variants share weights, calibration set, and refit step, so differences come only from how time enters the importance score.

The strongest data-driven competitor is SparseSSM's second-order procedure SparseSSM, run on irregular calibration sequences with the gaps fed through the model. We run it twice: with its power-law prefix weighting, and with uniform weighting. Alongside it we run empirical leave-one-mode-out ablation: each mode is removed in turn and the change in output on the calibration set is measured. All data-driven methods get the same calibration set at three sizes (64, 512, 4096 sequences) and the same number of hyperparameter trials.

The training-based reference points are a model trained from scratch at the target state size, and an HSVR-regularized model Hankel Singular Value Regularization for Highly Compressible State Space Models trained on the irregular data and then truncated. Both receive the full training budget of the dense model. We report them as costing a training run rather than a one-shot pass.

For the dense task baselines, the file gives these reported numbers. Neural Controlled Differential Equations for Irregular Time Series reports 0.880 AUC on PhysioNet sepsis prediction and 98.6% accuracy on CharacterTrajectories with 70% of observations dropped (ODE-RNN 95.3%, GRU-D 91.9%). From 805 ms to 23 ms reports 0.66 AUROC on PhysioNet 2012 in-hospital mortality. For any pruning method on irregular time series, no reported result was found on Lacuna. Beating a compression baseline means a higher task metric at equal kept-state fraction, without retraining, averaged over at least three trained models with confidence intervals. We additionally require that our dense model sit within a reasonable margin of these published dense numbers, so that gains are not measured on a weak starting point.

Why not just…?

Why not just resample to a regular grid and use an existing one-shot pruner? Gridding throws away the gap information that makes fast and slow modes differ, and forward-filling erases the signal carried by when a test was ordered. Even on regular grids, SparseSSM reports that standard pruners collapse on SSMs past about 30% sparsity. We include a gridded-and-pruned arm on all benchmarks, so this is settled empirically rather than argued.

Why not just measure each mode's importance empirically on irregular calibration data? This is the real threat. Leave-one-mode-out ablation, or SparseSSM run on irregular sequences, automatically reflects the true gap distribution, and it may match our criterion when calibration data are plentiful and drawn from the deployment regime. We claim an advantage in two places. The first is small calibration sets, since the closed form estimates a smooth function of gaps instead of noisy per-mode output differences. The second is regime shift: our criterion can be recomputed from a new site's timestamps alone, without running the model on that site's data. The calibration-size sweep and the cross-regime test below decide this. If empirical ablation matches us everywhere, we will report that the contribution is the finding that pruning must be calibrated under the deployment gaps, not the formula.

Why not just train a small model? COMPRESSM's 85.7% versus 81.8% on LRA CIFAR-10 suggests large-then-pruned wins, but that evidence comes from regular sequences. We rerun the comparison on irregular data and charge the from-scratch arm its full training cost.

Why not just distill into a small model? Distillation needs a second full training run. That is exactly what one-shot compression avoids, and the cost is worse for continuous-time teachers that are already slow. We treat it as out of budget and do not claim to beat it in accuracy.

Proposed Experiments

The benchmarks are CharacterTrajectories from the UEA archive with random observation drop rates of 30%, 50%, and 70%, plus a synthetic variant with bursty gaps drawn from a renewal process; PhysioNet 2012 mortality and PhysioNet sepsis prediction (AUROC); and MIMIC-III 25-task phenotyping (macro AUROC). For latency, we report model time and end-to-end time including preprocessing on the same GPU. Following From 805 ms to 23 ms, we state plainly how much of the total is reachable by state pruning at all.

Four checks are decisive.

The first is the cross-regime test. We prune using timestamps from the 30%-drop regime and deploy at 70%, and the reverse. We compare that against recomputing our criterion from the deployment timestamps only, and against empirical ablation calibrated on the wrong regime. The hypothesis fails if retargeting from timestamps recovers no more than a small fraction of the gap between mismatched and matched calibration.

The second tests the predicted direction of bias. On bursty synthetic gaps, the mean-gap surrogate should discard fast modes that our criterion keeps, and accuracy should favor our criterion. On regular-but-sparse gaps, the two should nearly agree. Seeing an advantage where Jensen predicts none, or none where it predicts one, would mean our explanation is wrong even if accuracy improves.

The third is the informative-missingness test on PhysioNet. If the independence estimator underperforms the joint estimator there, but not on synthetic data with random drops, the independence assumption is the failure point. That would be a useful, reportable result.

The fourth is the prefix-weighting test. Comparing SparseSSM with and without power-law prefix weighting on sepsis prediction, where onset occurs late, checks whether position-based weighting hurts on clinical data, as the conflict between its assumption and late clinical events suggests.