Robust and Nonstochastic Control Under Partial Observability

30 concepts31 papers1 proposal

Control theory and reinforcement learning have recently achieved remarkable theoretical milestones. We now possess algorithms capable of navigating completely unpredictable, adversarial environments with sublinear regret, as well as frameworks that guarantee long-horizon safety under severe distribution shifts. However, a quiet, pervasive assumption underpins many of these breakthroughs: full state observability.

In a fully observable system, the controller has perfect, instantaneous access to the exact internal state vector of the environment. But real-world engineering tells a different story. Sensors are noisy, hardware fails, and critical physical variables remain latent. When controllers are forced to operate based on incomplete or indirect sensor data, the elegant theoretical guarantees of fully observable models frequently collapse.

Across robust control, online optimization, and continuous-time reinforcement learning, researchers are actively working to bridge this "partial observability gap."


Why the Separation Principle Fails

Historically, control theory bypassed the partial observability problem by relying on the Separation Principle. In standard linear systems subject to clean Gaussian noise, you can mathematically split the control problem into two independent parts:

  1. Estimate the most likely state of the system using an observer (like a Kalman filter).
  2. Compute the optimal control inputs using a framework like the Linear Quadratic Regulator (LQR), acting as if your state estimate is perfectly correct.

However, modern applications demand controllers that can withstand non-Gaussian noise, adversarial disturbances, and structural model misspecifications. In these complex regimes, the Separation Principle breaks down entirely.

As Jun Morimoto et al. noted in their foundational work on Robust Reinforcement Learning (RRL), treating robust learning as a differential game yields highly stable policies, but these frameworks heavily rely on the assumption that all system states are perfectly observable. When we move to real-world robotics where sensors provide limited or noisy feedback, estimation and control become tightly coupled. Your actions change what you can observe, and your incomplete observations distort your actions.


The Breakdown of Online and Nonstochastic Control

The tension between full state assumptions and practical deployment is particularly evident in nonstochastic control, which frames control as online convex optimization with memory.

Algorithms like those introduced by Naman Agarwal et al. in Online Control with Adversarial Disturbances achieve near-optimal regret bounds against adversarial disturbances. Similarly, Asaf B. Cassel et al. achieved optimal regret rates in Bandit Linear Control.

Why do these algorithms struggle when transitioned to partially observable settings?

To minimize regret against an unpredictable environment, a nonstochastic controller typically uses a disturbance-action policy, choosing its next input based on a history of past environmental disturbances. However, if the controller only receives noisy or indirect measurements, it cannot perfectly isolate or reconstruct those past disturbances.

[Adversarial Disturbance] ---> [Hidden System State] ---> [Noisy Sensor Output]
                                                                  |
[Disturbance-Action Policy] <--- CANNOT PERFECTLY RECONSTRUCT <---+

As highlighted by Nanfei Jiang et al. in Online Nonstochastic Control with Convex Safety Constraints, extending nonstochastic control to partially observable environments remains a significant open challenge because sensor noise becomes mathematically entangled with environmental disturbances. This same limitation applies to iterative frameworks: the requirement for perfect state observation to identify disturbances limits the real-world utility of methods like the Iterative Gradient Perturbation Controller proposed by Naman Agarwal et al. in A Regret Minimization Approach to Iterative Learning Control.


Bridging the Gap via Distributional Robustness

To design controllers that work without full state information, researchers are turning to Distributionally Robust Control. Instead of assuming a fixed, perfectly known noise distribution, these frameworks optimize performance against the worst-case distribution within a specific uncertainty set, often defined using the Wasserstein metric.

Initially, even these distributionally robust frameworks assumed full state observability, as discussed by Kihyun Kim 0001 et al. in Minimax control of ambiguous linear stochastic systems using the Wasserstein metric. The breakthrough comes from reformulating the minimax optimization problem to accept output feedback rather than direct state feedback.

In Wasserstein Distributionally Robust Control of Partially Observable Linear Systems: Tractable Approximation and Performance Guarantee, Astghik Hakobyan et al. demonstrated how to approximate the Wasserstein distance over hidden states using a tractable penalty term. By leveraging the Gelbrich bound, the authors in Wasserstein Distributionally Robust Control of Partially Observable Linear Stochastic Systems derived closed-form optimal policies that operate purely on noisy sensor measurements.

By optimizing over a cloud of plausible state distributions rather than a single point estimate, these controllers inherently account for the measurement uncertainty caused by partial observability. This distributional approach has been successfully extended to minimize regret over infinite horizons by Taylan Kargin et al. in Wasserstein distributionally robust regret-optimal control over infinite-horizon, providing a mathematical bridge between state estimation and robust optimization.


The Continuous-Time and Safety Frontiers

The challenge of partial observability intensifies when moving to continuous-time systems governed by Stochastic Differential Equations (SDEs) or systems requiring strict safety guarantees.

  • Continuous-Time Reinforcement Learning: In continuous-time domains, Qian Qi proved the convergence of Deep Q-Networks in Universal Approximation Theorem of Deep Q-Networks, but emphasized that extending such continuous-time proofs to partially observable Markov decision processes increases the underlying mathematical complexity exponentially. When exploring unknown continuous-time dynamics, agents must balance exploration and exploitation. Mohamad Kazem Shirani Faradonbeh et al. developed efficient online learning algorithms in Online Reinforcement Learning in Stochastic Continuous-Time Systems, yet they explicitly note that transitioning from full state feedback to partial observability is the critical next step for real-world deployment.
  • Long-Horizon Safety: When maintaining safety is the primary objective, frameworks like Lyapunov Density Models—introduced by Katie Kang et al. in Lyapunov Density Models: Constraining Distribution Shift in Learning-Based Control—are designed to keep agents within safe training distributions. However, because these methods assume full state observability, developing mechanisms that provide long-horizon safety guarantees under partial observability remains an active and vital area of research.

Ultimately, the transition away from the full state observability assumption represents a maturation of the field. By treating state estimation not as an isolated preprocessing step, but as an integrated component of robust, nonstochastic, and distributionally robust optimization, current research is establishing the theoretical foundations necessary to deploy learning-based control safely into the messy, partially observable real world.

Key Foundational Work

Robust Reinforcement Learning (2000) — contributed 1 concept elements to this direction.
Linear-time temporal logic control of discrete event models of cooperative robots (2008) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_full_state_observability_assumption_d3d94b22,
  title = {full state observability assumption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/full-state-observability-assumption/rdr_d3d94b224eb249e1834084005d440f12},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_full_state_observability_assumption_d3d94b22_v1,
  title = {full state observability assumption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/full-state-observability-assumption/rdr_d3d94b224eb249e1834084005d440f12/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake