State-space models (SSMs) are the standard framework for tracking dynamic systems, from autonomous vehicles to evolving biological populations. Traditionally, Sequential Monte Carlo (SMC) methods—commonly known as particle filters—have been the tool of choice for performing inference in these models. However, a fundamental tension exists between particle filters and modern deep learning: the discrete, stochastic nature of particle filters makes them incompatible with the gradient-based optimization required for neural networks.
Recent research has focused on resolving this tension by creating Differentiable Particle Filters (DPFs). By replacing non-differentiable operations with smooth approximations, researchers have enabled the end-to-end training of complex tracking systems, allowing neural networks to learn transition dynamics and observation models directly from raw, high-dimensional data.
The Differentiability Barrier
A standard particle filter represents a probability distribution as a set of discrete "particles." At each time step, the filter performs two operations that typically break gradient flow:
- Sampling: Generating new particle states from a proposal distribution.
- Resampling: Selecting a new set of particles based on their importance weights to prevent "particle collapse," where a few particles dominate the distribution.
As Hsu et al. note, these discrete steps create discontinuities in the computational graph. If you change a model parameter slightly, the specific particles chosen during resampling might jump abruptly, making the resulting loss function non-differentiable and preventing the use of backpropagation.
Reparameterizing the Particle Cloud
The first step in making these filters differentiable is addressing the sampling process. Drawing inspiration from Variational Autoencoders, many DPFs employ the reparameterization trick.
Wen et al. and Gama et al. demonstrate that by expressing latent state transitions as deterministic functions of the previous state, model parameters, and an external fixed noise source, the sampling step becomes differentiable. This allows gradients to flow from the likelihood of an observation back through the generated particles to the underlying neural network parameters.
Smoothing the Resampling Step
The more difficult challenge is the resampling step, which is inherently categorical. Researchers have proposed several distinct strategies to overcome this:
- Soft Resampling: Hsu et al. introduced the Particle Filter Network (PF-net), which replaces discrete selection with a differentiable approximation based on importance sampling. This maintains a continuous relationship between particle weights and the resulting distribution.
- Optimal Transport: A more principled approach involves treating resampling as a transport problem. Corenflos et al. and van Nierop et al. utilize entropy-regularized Optimal Transport (the Sinkhorn algorithm). Instead of discarding low-weight particles, the algorithm "transports" the particle cloud toward high-probability regions in a smooth, differentiable manner.
- Common Random Numbers: Rosato et al. argue that by using common random numbers to fix the stochasticity of the selection process, one can obtain piece-wise continuous gradients. This approach is often more computationally efficient than transport-based methods while remaining sufficient for gradient-based samplers like NUTS.
Bridging Sampling and Variational Inference
Making particle filters differentiable does more than just enable tracking; it bridges the gap between sampling-based methods and Variational Inference (VI). Naesseth et al. show that SMC can provide a differentiable estimate of the marginal likelihood, which serves as a tighter Evidence Lower Bound (ELBO) for training latent variable models.
This integration is particularly powerful for complex structures. For instance, Lai et al. propose Variational Marginal Particle Filters, which use Rao-Blackwellization to marginalize out discrete variables, enabling the use of implicit reparameterization gradients. Similarly, Chen et al. apply these differentiable bounds to the "super-exponential" search space of Bayesian phylogenetics, allowing for the optimization of evolutionary tree topologies via gradient descent.
Beyond Filtering: Differentiable MCMC
The quest for differentiability extends to Markov Chain Monte Carlo (MCMC) methods as well. Traditional MCMC algorithms like Hamiltonian Monte Carlo (HMC) rely on a Metropolis-Hastings "accept-reject" step, which is non-differentiable.
Geffner and Domke and Zhang et al. propose removing these discrete corrections to create Uncorrected Hamiltonian Annealing. By treating the entire sampling trajectory as a differentiable function, they enable the joint optimization of the sampler's internal dynamics—such as step sizes and annealing schedules—alongside the model parameters. As Zoltowski and Cai demonstrate, rejection-free methods like slice sampling can similarly provide low-variance reparameterization gradients for unnormalized probability distributions.