Exact Count Filtering at Any Sampling Rate

Many count time series are recorded at uneven intervals: animal surveys run when weather allows, case reports arrive weekly in some regions and daily in others, and molecule counts are taken at whatever times an experiment permits. The best exact methods for hidden integer counts, such as an unobserved population size behind noisy census data, are built for a fixed time step. They represent each distribution by its probability generating function (PGF), a power series whose coefficients are the probabilities, and chain these functions across steps without capping the count at an arbitrary maximum. Change the spacing between observations and the step-to-step kernel no longer means anything. Neural sequence models solved the analogous problem for real-valued signals. They learn a continuous-time system and discretize it with whatever gap $\Delta$ separates two observations, so the same parameters work at any sampling rate. That machinery only handles real vectors, though, not integer counts with unbounded support.

We propose to give the exact PGF filter the same property. For the linear birth–death–immigration processes behind the integer hidden Markov models in this literature, the transition generating function over an interval of any length $\Delta$ has a closed form, and successive intervals compose exactly. Built into an exact forward filter, this gives a count model that, by construction, gives the same answer whether data are recorded hourly, daily, or at random times. It still needs no truncation of the state space. It can also produce a full forecast count distribution at any horizon from the same formula. The main claim to test is this: one closed-form evaluation per observation gap matches the accuracy of the strongest truncated alternative. It scales with the observed counts rather than with a chosen ceiling, and it avoids the numerical blow-up that repeated composition causes over long gaps.

Background and the gap

Exact PGF inference for integer latent-variable models was introduced by Exact Inference for Integer Latent-Variable Models. Its GDUAL-FORWARD algorithm evaluates nested generating functions and their high-order derivatives numerically, using Taylor-polynomial arithmetic. Its cost grows with the observed counts rather than a truncation bound, and it reports up to an 8x speedup over truncation when detection probability is low. Learning in Integer Latent Variable Models with Nested Automatic Differentiation added exact likelihood gradients through nested automatic differentiation. It also added a logarithmic number system, which stores the sign and the log magnitude of each value, because ordinary 64-bit arithmetic overflowed once the immigration rate exceeded 100. Exact Bayesian Inference on Discrete Models via Probability Generating Functions generalized the approach into a probabilistic programming language with the Genfer engine. These three papers supply our method. All three treat time as an index $t \to t+1$, so an observation gap of 2.7 units has no meaning in them.

On the other side, Efficiently Modeling Long Sequences with Structured State Spaces and Simplified State Space Layers for Sequence Modeling show that learning a continuous-time linear system and discretizing it by $\Delta$ gives zero-shot transfer across sampling rates and native handling of irregular gaps. FlowState carries this to forecasting foundation models by decoding to continuous Legendre-polynomial trajectories. These papers motivate the proposal: the step-size-as-parameter idea works. They also show its limit, because their outputs are real numbers that can be negative or fractional, and they say nothing about integer support. A study on discrete-time movement models documents the failure we target: a discrete-time model fit at one step size confuses a change in observation rate with a change in dynamics.

Two papers constrain the design. Interpretable and tractable models of transcriptional noise shows that two transcription-noise models agree on their low-order moments and are only told apart by full discrete distributions. That evidence argues against continuous moment-matching relaxations. The same paper notes that parameters become unidentifiable in Poisson-like regimes, which warns us that some rates may not be recoverable from sparse sampling. We have to measure this rather than assume it away.

Core mechanism

Consider a hidden count $N(t)$ that evolves in continuous time. Each individual gives birth at rate $\lambda$ and dies at rate $\mu$, and new individuals immigrate at rate $\nu$. We observe binomially thinned counts $Y_k \sim \text{Binomial}(N(t_k), \rho)$, where $\rho$ is the detection probability, at arbitrary times $t_k$. Over a gap $\Delta$, each individual alive at the start leaves descendants whose generating function $\phi_\Delta(z)$ is a closed-form ratio of linear functions of $z$ (a Möbius map, the classical Kendall solution). Immigrants arriving during the gap contribute an independent factor $\psi_\Delta(z)$. The prior over $N(t_k)$ given observations up to $t_{k-1}$ then follows from the filtered generating function $F_{k-1}$:

$$ P_k(z) = F_{k-1}\big(\phi_{\Delta_k}(z)\big)\,\psi_{\Delta_k}(z), \qquad \Delta_k = t_k - t_{k-1}. $$

In the pure survival-plus-immigration case ($\lambda = 0$), the descendant generating function is $\phi_\Delta(z) = 1 - e^{-\mu\Delta}(1-z)$. The immigration factor $\psi_\Delta$ is Poisson with mean $\nu(1-e^{-\mu\Delta})/\mu$. This is the continuous-time version of the survival and immigration kernels used by Winner et al. Because these processes are time-homogeneous Markov processes, the semigroup identity $\phi_{\Delta_1+\Delta_2} = \phi_{\Delta_1}\circ\phi_{\Delta_2}$ holds exactly. Sampling-rate invariance is therefore a property of the model rather than something learned.

The observation update, the binomial thinning plus the conditioning on $Y_k$, is unchanged from GDUAL-FORWARD, so the whole filter reuses existing Taylor-polynomial arithmetic. A forecast at any horizon $h$ is simply $F_K(\phi_h(z))\,\psi_h(z)$. Its coefficients give the exact predictive count distribution, so the method also serves as a decoder that can be queried at any time, which is the count analogue of FlowState's continuous decoder.

The dynamic-range concern can be stated precisely. A discrete-step model forced onto a long gap must compose its kernel many times. The number of compositions grows with $\Delta$, and the effective immigration grows as $\nu\Delta$, which is the regime where Sheldon et al. saw overflow. The closed form needs one evaluation per gap regardless of $\Delta$. We expect the cost per observation to stay flat as gaps widen, while repeated composition grows linearly in the number of base steps. Whether logarithmic-number-system storage is still needed at large $\nu\Delta$ is an empirical question we will answer rather than assume.

Research Program

The first deliverable is a filter for irregular gaps. It covers survival–immigration, linear birth–death–immigration, and Poisson branching within each interval. It provides exact marginal likelihoods, and it provides gradients with respect to $(\lambda, \mu, \nu, \rho)$ by differentiating through $\phi_\Delta$ and $\psi_\Delta$ with the forward-over-reverse scheme of Sheldon et al.

The second deliverable is rates that vary over time. We let the rates be piecewise constant between observations. The closed form survives this, because each interval is time-homogeneous. Optionally, a small continuous-time encoder in the style of S5 can produce these rates from covariates, which gives a neural model with exact count likelihoods. We treat this as an extension, not the core claim. Its only job is to check that the exact likelihood stays usable as a training objective.

The third deliverable is a numerical study. It maps where float64, logarithmic-number-system storage, and the closed form each fail, as functions of $\nu\Delta$, the observed counts $Y$, and $\rho$.

We do not attempt backward smoothing or multivariate coupled counts. Both remain open in the PGF literature, and folding them in would dilute the test.

Baselines

The naive reference point is the fixed-step integer hidden Markov model of Winner et al.. It is fit at one sampling rate with its per-step kernel and applied unchanged when the rate changes. This is the time-step mismatch, and it shows how much invariance is worth. A stronger version of the same idea serves as the key ablation. When gaps are integer multiples of a base step, it composes the discrete kernel $k$ times, with and without logarithmic-number-system storage as in Sheldon et al.. Comparing it against our closed form, on identical data and parameters, isolates the single ingredient we add: one closed-form evaluation per gap instead of repeated composition. Beating it means equal likelihood to machine precision, with cost that stays flat as gaps grow and no overflow where composition overflows.

The strongest competitor is truncation of the continuous-time generator. We cap the count at $C_{\max}$, form the finite rate matrix, and apply its matrix exponential for each gap $\Delta_k$. This handles arbitrary gaps directly and is exact up to the cap, which makes it the real threat. We run it with $C_{\max}$ increased until the likelihood converges to $10^{-8}$, so its accuracy is not handicapped, and we report wall-clock time at that cap on the same hardware. On fixed-step data, Winner et al. report up to an 8x speedup of GDUAL-FORWARD over truncation in low-detection regimes. For truncated matrix exponentials on irregular gaps, we found no reported result on Lacuna. Beating this baseline means matching its converged likelihood while being faster in the regimes where the required cap is large (low $\rho$, bursty counts). We will report honestly where it wins.

A sequential Monte Carlo particle filter for the same continuous-time process serves as the sampling baseline. It gets matched wall-clock budgets, and we report likelihood variance and bias against ground truth. Genfer reports runtimes competitive with such samplers alongside zero approximation error (Zaiser et al.), but on fixed-step programs.

A continuous relaxation serves as the real-valued baseline. An S5 model is trained on log-counts with Gaussian output, using the same irregular timestamps. FlowState reports a mean absolute scaled error of 0.701 on GIFT-Eval (FlowState). That number is aggregate over mostly real-valued series and is not comparable to count log-likelihoods, so we rerun the S5 model on our tasks. We score it with a discretized predictive distribution. We count our method as beating it if it achieves better held-out log predictive probability and calibration, particularly at sampling rates unseen during fitting.

Why not just…?

Why not just truncate the generator and take matrix exponentials? This is the alternative most likely to match us, and we treat it as the main comparison rather than a straw man. It handles irregular gaps exactly up to the cap. Its weakness is that cost scales with $C_{\max}$ rather than with the data: the exponential of an $N\times N$ matrix is roughly cubic in $N$. Low detection probability forces a large cap, and the fixed-step evidence from Winner et al. shows the gap is real there. If the truncated exponential matches us across all tested regimes, including large counts and low $\rho$, then the contribution shrinks to exactness and memory. We would report that outcome plainly.

Why not just compose the fixed-step kernel as many times as the gap requires? This only works when gaps are integer multiples of a base step. The composition depth grows with the gap, which pushes high-order derivatives into the regime where Sheldon et al. saw overflow above an immigration rate of 100. The ablation described under Baselines measures exactly this cost and stability difference.

Why not just model log-counts with S5 or FlowState? These models inherit sampling-rate invariance for free. However, their outputs are not integer distributions, and Gorin et al. show that low-order statistics can fail to separate mechanisms that full count distributions distinguish. The test is held-out log predictive probability at small counts and low detection, where a Gaussian relaxation should be worst. If the relaxation is competitive at large counts, we will say that the exact method matters only in the small-count regime.

Why not just run a particle filter? It is general, but it gives noisy likelihoods, and that noise corrupts gradient-based fitting and model comparison. The matched-budget comparison will show whether its bias and variance are material at realistic particle counts.

Proposed Experiments

The synthetic suite rebuilds the integer hidden Markov model and population benchmarks of Winner et al. and Sheldon et al. in continuous time. It sweeps $\rho \in [0.1, 0.9]$ and immigration rates up to and beyond 100 per unit time. Observation times are drawn three ways: on a regular grid, as Poisson-process times, and as bursty clusters separated by long gaps. Ground-truth likelihoods come from a truncated exponential with a very large cap. We measure absolute log-likelihood error, wall-clock time per observation as a function of gap length and observed counts, and overflow incidence.

The decisive invariance check mirrors the cross-rate test on Speech Commands in Gu et al.. We fit parameters on series sampled at one rate. We then evaluate held-out log predictive probability and parameter recovery error on the same processes resampled at 0.25x, 0.5x, 2x, and 4x, and on irregular schedules. The fixed-step baseline should degrade, while our model should not degrade beyond what the information lost to sparser sampling explains. To separate those two effects, we compare against a model refit at each rate.

For real data, we use the coal-mining disaster switchpoint counts from the Genfer benchmarks. We re-aggregate them into randomly varying bins to check that inferred rates are stable across binnings. We do not rely on the steady-state single-cell data of Gorin et al., which has no time axis. Its unidentifiability warning motivates one more measurement: we compute the profile-likelihood width for $\lambda$ and $\mu$ as sampling becomes sparser, which shows when continuous-time rates stop being recoverable from count data at all.