Robust Control Under Partial Observability

234 concepts201 papers1 proposal

In control theory, the ideal scenario is full state feedback: the controller has immediate, noise-free access to every internal variable of the system—such as the exact position, velocity, and internal temperature of a drone. With this complete state vector, designing optimal or robust controllers is a well-understood mathematical problem, often reducing to elegant, convex optimizations.

In the real world, this full-state assumption is an illusion. Actuators have physical limits, environments are uncertain, and sensors are noisy or incomplete. A robot might only have a camera feed or a few boundary sensors rather than direct access to its underlying state variables. This mismatch forces us into the realm of output feedback control, where we must steer a system using only a history of corrupted, partial measurements.

To understand why output feedback is challenging—and how modern control theory solves it—we must look at how estimation and control interact, how data can substitute for hidden states, and how optimization landscapes change when visibility is limited.

The Breakdown of Separation

For linear systems under ideal conditions, a foundational concept known as the Separation Principle applies: you can design a state estimator (like a Kalman filter) and a controller independently, plug them together, and the combined system is guaranteed to be stable.

In robust, stochastic, or nonlinear control, this separation completely breaks down. The moment a controller relies on estimated states, a dangerous feedback loop emerges: the control actions affect the system's trajectories, which affects the sensor measurements, which in turn alters the estimation error.

This coupling introduces severe non-linearities and time lags. As Imsland et al. demonstrate in the context of nonlinear predictive control, the estimation lag introduced by an observer can easily destabilize an otherwise perfect controller.

Furthermore, there is a fundamental trade-off between control aggressiveness and estimation accuracy. If a controller reacts too violently to disturbances, it can amplify noise and degrade the observer's performance. For instance, Polyakov et al. studied linear plants with multiplicative disturbances and revealed that aggressive, high-gain relay control strategies actually degrade state estimation accuracy. Instead, the authors showed that smooth, continuous control laws perform significantly better because they avoid exciting unmodeled dynamics and noise loops within the observer.

+-----------------+     Inputs (u)     +---------------+
|                 |------------------->|               |
|   Controller    |                    |    System     |
|                 |<-------------------|               |
+-----------------+     Outputs (y)    +---------------+
         ^                                     |
         | (Noisy/Incomplete)                  v
         +----------------------------- [Hidden States (x)]

When internal states are hidden, we cannot simply treat the estimate as the ground truth. We must design the controller to be robust to the uncertainty of the estimate itself.

Bypassing States via Input-Output Histories

If estimating the exact internal state vector $x(t)$ is so fraught with difficulty, can we bypass it entirely?

For linear time-invariant (LTI) systems, the answer is yes, thanks to a powerful realization: a finite history of past inputs and outputs uniquely characterizes the current state of an observable system. Instead of mapping an explicit state estimate $\hat{x}(t) \to u(t)$, we can parameterize our controller to map a window of past measurements directly to the next control action:

$$\begin{pmatrix} y(t-T) \\ \dots \\ y(t) \end{pmatrix}, \begin{pmatrix} u(t-T) \\ \dots \\ u(t-1) \end{pmatrix} \longrightarrow u(t)$$

This perspective underpins modern data-driven control frameworks. When a mathematical model of a system is unavailable, we must learn to control it purely from experimental trajectories. However, as Berberich et al. point out, moving from state feedback to output feedback in data-driven regimes requires strict structural assumptions, such as knowing the system's observability and controllability indices. If these indices or the internal lag structure are complex or unknown, providing end-to-end stability guarantees becomes significantly harder.

Despite these hurdles, recent breakthroughs show that output-feedback systems can achieve the same fundamental efficiency as full-state systems. By leveraging Input-Output Parameterization (IOP), Zheng et al. analyzed the sample complexity of learning a Linear Quadratic Gaussian (LQG) controller from a single noisy trajectory. The authors proved that the performance degradation scales linearly with model estimation error. Crucially, this matches the theoretical bounds of fully observed systems, proving that hidden states do not inherently doom a data-driven controller to poor sample efficiency.

When we cannot afford the computational complexity of tracking long histories of inputs and outputs, we often turn to Static Output Feedback (SOF). In SOF, the control input is calculated by multiplying the current measurement vector by a static matrix: $u(t) = Ky(t)$.

While computationally trivial to implement on hardware, SOF is notoriously difficult to design. In full state feedback ($u = Kx$), the optimization landscape for finding an optimal $K$ is often convex or possesses "convex-like" properties, meaning any local minimum is a global minimum. In SOF, because $y = Cx$, the closed-loop system matrix becomes $A + BKC$. The control matrix $K$ is trapped between the input matrix $B$ and the output matrix $C$. This structural sandwich warps the optimization landscape, making the problem highly non-convex and NP-hard in the general case.

State Feedback:       A + BK       (Convex-like optimization)
Static Output:        A + BKC      (Non-convex, trapped by C)

This non-convexity has long frustrated researchers trying to apply model-free reinforcement learning or policy gradient methods to output feedback problems. Without global convergence guarantees, a gradient descent algorithm might get stuck in a poor local minimum, leading to unstable control behavior.

However, recent mathematical analysis shows that these landscapes are more forgiving than they appear. Duan et al. investigated the optimization landscape of policy gradient methods for discrete-time SOF. The authors proved that despite non-convexity, policy gradient variants converge to stationary points at a dimension-free rate and exhibit linear convergence when initialized near a local minimum.

Similarly, Guo et al. revisited the classic $\mathcal{H}_\infty$ robust control problem using direct policy search. They demonstrated that even though the robust output-feedback optimization problem is both non-convex and non-smooth, specialized subgradient methods can still find globally optimal solutions under certain conditions.

Why It Works: The Intuition of Bounded Uncertainty

Ultimately, output feedback control works because it shifts the goalpost from eliminating uncertainty to bounding it.

Whether you are using a continuum backstepping observer to estimate the internal states of a partial differential equation from boundary data (as explored by Liu), or using Sum-of-Squares optimization to map high-dimensional camera images directly to stable quadrotor maneuvers (as developed by Chou and Tedrake), the core mechanism remains the same:

  1. Quantify the Estimation Error: Treat the difference between the true state and the observed proxy as a bounded disturbance.
  2. Design for Worst-Case Bounds: Ensure the controller is robust enough to handle any state variation within that error bound.
  3. Smooth the Control Action: Avoid aggressive, high-frequency control inputs that degrade sensor readings or violate physical actuator constraints.

By treating partial observability not as a fatal flaw, but as a structured source of bounded noise, modern output feedback control bridges the gap between clean theoretical models and the messy reality of physical hardware.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 186 papers
  2. Version 1 · 2026-10-04 · 251 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_output_feedback_control_b24c2a55,
  title = {output feedback control},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/output-feedback-control/rdr_b24c2a55631844438ff2c9ecd53d4194},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_output_feedback_control_b24c2a55_v2,
  title = {output feedback control},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/output-feedback-control/rdr_b24c2a55631844438ff2c9ecd53d4194/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake