Duality Between Filtering and Optimal Control

50 concepts24 papers1 proposal

At first glance, state estimation and optimal control appear to be opposite endeavors. In state estimation (filtering), we observe noisy outputs of a system and try to reconstruct its hidden internal state. In optimal control, we inject inputs into a system to steer its state toward a desired target.

Yet, these two problems are deep mathematical mirrors of one another. This relationship, known as estimation-control duality, establishes that solving a filtering problem is mathematically equivalent to solving an optimal control problem in reverse time, and vice versa.

Forward Time:  [Initial State] ---> (Noisy Dynamics) ---> [Estimated State]  (Filtering)
                                      ▲             ▼
Backward Time: [Target State]  <--- (Control Law) <--- [Optimal Input]     (Control)

By viewing filtering through the lens of control—or control through the lens of filtering—we gain powerful algorithmic tools. We can bypass the curse of dimensionality in reinforcement learning, optimize filters without knowing underlying noise statistics, and provide rigorous variational foundations for particle-based estimation.


The Mechanics of the Mathematical Mirror

Why does this duality exist? Consider the classical Linear Quadratic Regulator (LQR) and the Kalman Filter. The LQR seeks a control policy to minimize a quadratic cost of state deviations and control efforts over time. The Kalman Filter seeks a gain matrix to minimize the variance of the state estimation error.

Both problems are governed by the exact same underlying mathematical engine: the Riccati differential equation. By constructing an adjoint system that runs backward in time, the initial uncertainty of an estimation problem maps directly onto the terminal cost of a control problem, while the measurement noise maps onto the control penalty.

This structural symmetry extends far beyond standard linear systems:

  • Differential-Algebraic Equations (DAEs): For systems with algebraic constraints (where states must satisfy static equations alongside differential ones), Mihály Petreczky demonstrates in Infinite horizon control and minimax observer design for linear DAEs that a generalized Kalman duality transforms complex observer design into a stable, dual optimal control problem, even when the system lacks unique forward solutions.
  • Nonlinear Markov Chains: To move beyond linear systems, Jin-Won Kim et al. established a nonlinear duality framework in An Approach to Duality in Nonlinear Filtering. By defining a backward ordinary differential equation (ODE) as the dual system to a forward filtering process, the authors showed that the minimum variance estimator for continuous-time Markov chains can be derived directly as an optimal control solution.

Solving Control Problems via Filtering

One of the most profound applications of this duality is rewriting stochastic optimal control and reinforcement learning (RL) as state estimation tasks.

Traditional model-free RL and control methods (such as path integral control) often rely on trajectory weighting via importance sampling. In high-dimensional systems, this leads to "weight collapse," where a few trajectories dominate the sample pool, causing exponential sample complexity.

By exploiting duality, we can instead treat the optimal control law as a hidden state to be tracked, allowing us to deploy data-assimilation tools like the Ensemble Kalman Filter (EnKF). In Ensemble Kalman Filter (EnKF) for Reinforcement Learning (RL), Anant A. Joshi et al. introduce a dual EnKF approach that simulates an ensemble of particles backward in time. Instead of weighting trajectories based on rewards, the particles interact and update their positions using a gain matrix derived directly from the ensemble's statistical variance.

Traditional Sampling:  [Trajectories] ---> Weighting by Reward ---> [High Variance / Collapse]
Ensemble Filtering:    [Particles]    ---> Interacting Variance ---> [Deterministic Convergence]

This approach fundamentally changes how algorithms scale. In Error Analysis of Sampling Algorithms for Approximating Stochastic Optimal Control, Amirhossein Taghvaei et al. prove that while importance-sampling methods suffer exponentially from the curse of dimensionality, this interacting particle system scales linearly with dimension. This makes it highly effective for complex, high-dimensional systems. For instance, Mouhacine Benosman in A Dual Ensemble Kalman Filter Approach to Robust Control of Nonlinear Systems: An Application to Partial Differential Equations successfully applies a dual EnKF framework to approximate robust control policies for systems governed by partial differential equations (PDEs) without requiring explicit gradient-based policy iterations.

As surveyed by Amirhossein Taghvaei and Prashant G. Mehta in A Survey of Feedback Particle Filter and related Controlled Interacting Particle Systems (CIPS), this duality allows data assimilation techniques to bypass traditional policy optimization loops entirely, learning optimal control gains in a single pass.


Designing Filters via Policy Optimization

The duality mirror works both ways. We can also use optimization and control theory to solve notoriously difficult estimation problems—specifically, learning optimal filters when the underlying system noise covariances are completely unknown.

Classically, computing the optimal steady-state Kalman gain requires exact knowledge of process and measurement noise matrices. If these are unknown, practitioners must resort to heuristic tuning. However, if we treat the filter gain as a control policy, we can optimize it directly from noisy observation data using gradient descent.

The reason this works so well lies in the inherited geometry of the control landscape. In Data-driven Optimal Filtering for Linear Systems with Unknown Noise Covariances, Shahriar Talebi et al. leverage LQR-filtering duality to prove that the optimization landscape of a filter gain is well-behaved and free of spurious local minima. This structural property guarantees that Stochastic Gradient Descent (SGD) will converge globally to the true optimal Kalman filter using only output data.

In a companion work, Duality-Based Stochastic Policy Optimization for Estimation with Unknown Noise Covariances, the authors extend this to provide non-asymptotic global convergence guarantees, proving that the filter gain optimization problem inherits the benign landscapes typical of linear quadratic control.

Expanding this to finite timelines, Haoran Li in Policy optimization of finite-horizon Kalman filter with unknown noise covariance reformulates the finite-horizon Kalman filter as a policy optimization task. By establishing a direct link between the optimal Kalman gain and the minimizer of a data-driven cost function, the author demonstrates that policy gradient techniques can efficiently learn time-varying filter parameters without explicit system identification.


Variational Foundations and Feedback Particle Filters

The intersection of control and estimation has also revolutionized nonlinear filtering through the development of the Feedback Particle Filter (FPF).

In traditional particle filtering, particles are propagated forward, and their importance weights are updated based on observations. This suffers from particle degeneracy in high dimensions. The FPF replaces weight updates with a feedback control law that actively steers particles toward the true posterior distribution.

Traditional Particle Filter: [Particle] ---> Propagate ---> Weight Adjustment (Degeneracy Risk)
Feedback Particle Filter:    [Particle] ---> Propagate ---> Controlled Steering (Stable)

Duality formalisms provide the mathematical justification for why this steering works. In Derivation and Extensions of the Linear Feedback Particle Filter Based on Duality Formalisms, Jin-Won Kim et al. show that the FPF control law can be rigorously derived by reformulating the filtering problem as a finite-horizon, deterministic linear-quadratic optimal control problem for a dual process running backward in time.

This control-centric view of filtering reveals that tracking a probability distribution is fundamentally an optimization problem. Richard S. Laugesen et al. prove in Poisson's Equation in Nonlinear Filtering that the evolution of the posterior distribution is a gradient flow (steepest descent) in the space of probability densities, governed by a Poisson equation. This variational foundation ensures that the FPF control law is exact, matching the true posterior perfectly.

Furthermore, this framework can be adapted to non-Euclidean spaces. Chi Zhang et al. extend the FPF to matrix Lie groups in Feedback particle filter on matrix lie groups, allowing the filter to respect geometric constraints for applications like robotic rotation and attitude estimation.


Generalizing the Duality Paradigm

The power of estimation-control duality is not restricted to standard Gaussian noise or unconstrained systems. It serves as a general mathematical bridge across various uncertainty frameworks:

  • Worst-Case (Minimax) Estimation: When noise is bounded rather than probabilistic, Jason Frank et al. demonstrate in Data Assimilation for Linear Parabolic Equations: Minimax Projection Method that a generalized Kalman duality converts continuous-time minimax estimation for parabolic PDEs into an equivalent deterministic optimal control problem, providing robust state estimates under worst-case uncertainties.
  • Physical Constraints: Real-world states often have physical boundaries (e.g., non-negative chemical concentrations). In Minimum variance constrained estimator, Prabhat K. Mishra et al. use minimum variance duality to map a constrained stochastic estimation problem to a deterministic optimal control task. This allows standard optimization solvers to enforce constraints directly while maintaining stochastic performance.
  • Non-Probabilistic Uncertainty: Moving away from probability measures entirely, Jeremie Houssineau and Adrian N. Bishop show in Smoothing and Filtering with a Class of Outer Measures that formulating uncertainty via "possibility functions" (using supremums instead of integrals) yields closed-form forward and backward filtering equations identical to classical Bayesian results. This proves that optimization-based calculus can completely replace integration-based probability in state estimation.

By treating filtering and control as two facets of a single underlying optimization framework, this research direction provides a unified toolkit. Whether designing data-driven filters without noise statistics or scaling reinforcement learning to high-dimensional continuous spaces, the mathematical mirror of duality ensures that an advance in one domain directly illuminates the other.

Key Foundational Work

Infinite horizon control and minimax observer design for linear DAEs (2013) — contributed 2 concept elements to this direction.
Bearings-Only Tracking in the Plane (1998) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_estimation_control_duality_85c53899,
  title = {estimation-control duality},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/estimation-control-duality/rdr_85c53899ef374061b0feb15c675658cd},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_estimation_control_duality_85c53899_v1,
  title = {estimation-control duality},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/estimation-control-duality/rdr_85c53899ef374061b0feb15c675658cd/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake