Partially Observable RL: Benign Structures and Simple Generic Algorithms
Lists
Only you see this
Abstract
Partially observable Reinforcement Learning (RL) applications, where agents must make a series of decisions without complete knowledge of the underlying states of the system, are widespread. Partially observable RL can be notoriously difficult—well-known information-theoretic results show that learning partially observable Markov decision processes (POMDPs) requires an exponential number of samples in the worst case. However, this does not preclude the possibility of identifying rich subclasses of structured POMDPs for which learning remains feasible. This survey aims to offer a high-level overview of recent advancements in learning structured POMDPs. We will identify clean and practical problem structures that facilitate sample-efficient learning. Additionally, we will introduce simple and generic algorithms for learning POMDPs under different structural conditions. Finally, we will provide a unified view of these results under the framework of well-conditioned predictive state representation, which further reveals new tractable classes of partially observable problems along the way.
BibTeX
@inproceedings{liu2025partially,
author = {Qinghua Liu and Chi Jin},
title = {{Partially Observable RL: Benign Structures and Simple Generic Algorithms}},
booktitle = {Statistical},
year = {2025},
doi = {10.1214/25-sts1012},
}Loading related content...